feature.shape, Gives Current (x, y) x -total rows/features and y the current fields)This is a gap analysis for the following Dataset in SDE 64.
This is an insight of the Feature classes attribute Completeness. The report highlight the Gaps existing in the dataset in the context of attributes having Null fields. it gives numerical quantity of each attribute field (total null/un-filled fields) as well as a graphical represention with other columns of a feature class.
Below are the feature classes existing in SDE.DRG_SURFACE_GROUND_WATER_NETWORK.
• Surface Ground Water Chute
• Surface Ground Water Culvert
• Surface Ground Water Culvert Wall
• Surface Ground Water Ditch
• Surface Ground Water Micro Tunnel Manhole
• Surface Ground Water Pond
• SGW Pumping Station Catchment Boundary
• Surface Ground Water Swale
• Surface Ground Water Takamul Supply Zone Catchment Boundries
• Surface Ground Water Holding Tanks
• Surface Ground Water Trench
• Surface Ground Water Valve Chamber
• Surface Ground Water Culvert Pipes
• Surface Ground Water Duct
• Surface Ground Water Gully Connection
• Surface Ground Water Gully Connection Stub Pipe
• Surface Ground Water Micro Tunnel Sewer Pipe
• Surface Ground Water Rising Main
• Surface Ground Water Sewer
• Surface Ground Water Stub Pipe
• SGW Sewer Catchment Boundary
• Surface Ground Water Trench
• Borehole
• Surface Ground Water Catchbasin
• Surface Ground Water Cover
• Surface Ground Water Discharge Chamber
• Surface Ground Water Fittings
• Surface Ground Water Gully Connection End Cap
• Surface Ground Water Gully
• Surface Ground Water Internal Backdrop
• SGW Deep Injection well
• Surface Ground Water Manhole
• Surface Ground Water Micro Tunnel Manhole
• Surface Ground Water Outfall Chamber
• Surface Ground Water Pumping Station
• Surface Ground Water Stub End (End Cap)
• Surface Ground Water Holding Tanks
• Surface Ground Water Valve
• Surface Ground Water Valve Chamber
#import geopandas
#import fiona
import pandas as pd
from arcgis import GIS
import matplotlib.pyplot as plt
%matplotlib inline
Surface Ground Water Chute feature class has:
• Attribute Fields (Columns): 35
• Total Records (Rows): 6
Below is a representation of null values for each attribute field
Surface_Ground_Water_Chute = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_FOUL_SEWER_NETWORK\SDE.SWARENGB")
Surface_Ground_Water_Chute.head(2)
| OBJECTID | AREA_NAME | AREA_ENG_NAME | ASSIGNED_A | OFFICE_TEL | MOBILE_NO | E_MAIL | DEPT_NAME | SHAPE_LENG | GLOBALID | SUBTYPE_CODE | GFCODE | STATUS | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 7055 | Area A1 | Mr. Shakeel Ahmad | Area A1 | 7279 | 55594209 | shakeel@ashghal.gov.qa | Drainage O & M Management | 388671.988334 | {B5B5FEE5-0FC4-4A4E-AE1A-D99CD44710A2} | 1 | SWARENGB | Operational | None | None | None | None | 0 | None | None | None | None | None | 51.246785 | 24.877285 | None | None | None | None | None | None | None | None | None | {"rings": [[[231103.9626000002, 382786.9872999... |
| 1 | 7056 | Area B | Mr. Mohd Radzi Othman | Area B | 7179 | 55947632 | mrothman@ashghal.gov.qa | Drainage O & M Management | 399890.482856 | {9848891B-A606-467C-81A7-A0788E089840} | 1 | SWARENGB | Operational | None | None | None | None | 0 | None | None | None | None | None | 51.451025 | 25.740866 | None | None | None | None | None | None | None | None | None | {"rings": [[[203319.07579999976, 489262.407400... |
Surface_Ground_Water_Chute.shape
(6, 35)
Surface_Ground_Water_Chute_null = pd.DataFrame(Surface_Ground_Water_Chute.isnull().sum())
Surface_Ground_Water_Chute_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| AREA_NAME | 0 |
| AREA_ENG_NAME | 0 |
| ASSIGNED_A | 0 |
| OFFICE_TEL | 0 |
| MOBILE_NO | 0 |
| E_MAIL | 0 |
| DEPT_NAME | 0 |
| SHAPE_LENG | 0 |
| GLOBALID | 0 |
| SUBTYPE_CODE | 0 |
| GFCODE | 0 |
| STATUS | 0 |
| MXASSETNUM | 6 |
| MXLOCATION | 6 |
| MXSITEID | 6 |
| MXCREATIONSTATE | 6 |
| ROWSTAMP | 0 |
| MXSTATUS | 6 |
| PARENT_GLOBALID | 6 |
| LOCATION_GLOBAL_ID | 6 |
| DATA_LOAD_ID | 6 |
| DISCIPLINE | 6 |
| GPS_X | 0 |
| GPS_Y | 0 |
| GPS_Z | 6 |
| OPERATOR | 6 |
| MAINTAINER | 6 |
| MUNICIPALITY | 6 |
| DISTRICT | 6 |
| CONTRACT_ID | 6 |
| PROJECT_ID | 6 |
| ASSET_TAG | 6 |
| ASSET_PRIORITY | 6 |
| SHAPE | 0 |
x = Surface_Ground_Water_Chute.columns
y = Surface_Ground_Water_Chute.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Chute", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Culvert feature class has:
• Attribute Fields (Columns): 80
• Total Records (Rows): 1,208
Below is a representation of null values for each attribute field
Surface_Ground_Water_Culvert = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGARCLVT")
Surface_Ground_Water_Culvert.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | SUBTYPE_CODE | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | CALCLEN | SGW_TYPE | CVT_NUM | IN_EASTING | IN_NORTHING | IN_GND_LVL | IN_INV_LVL | OUT_EASTING | OUT_NORTHING | OUT_GND_LVL | OUT_INV_LVL | NO_OF_PIPES | DIAMETER | MATERIAL | CONST_DATE | MANUFACTURER | BEDDING | GROUND_TYPE | SLEEVED | INFILTRATION | WATER_TABLE | PIPE_CONDITION | PIPE_CLASS | REMARKS | PS_AREA | PROJ_REF | STATUS | TAKAMUL_CATCHMENT | CONFIDENCE | RELIABLE | GLOBALID | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 11 | Culvert | IA 12/13 C 063 ST | As Built | None | None | NaN | NaN | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-04-12 | VV | NaT | None | None | None | None | VV/BD | 2020-06-17 | None | None | Ashghal | None | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | 43.161867 | N | NOH-C4/RM31 | 212991.5355 | 409769.9539 | 19.72 | 18.79 | 212987.8754 | 409812.9603 | 19.98 | 18.51 | 1.0 | 1000.0 | GRP | 2017-07-29 | AMIANTIT | CS | 1 | N | N | B | 1 | 2 | None | None | C2014/14 | Connected / Live | Qatar West | 90 | 90.0 | {3F92F102-E3A4-4B70-977B-A95847415605} | {"hasZ": true, "rings": [[[212987.59350000042,... |
| 1 | 12 | Culvert | IA 12/13 C 063 ST | As Built | None | None | NaN | NaN | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-04-12 | VV | NaT | None | None | None | None | VV/BD | 2020-06-17 | None | None | Ashghal | None | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | 44.317332 | N | NOH-C4/RM32 | 213270.8699 | 409768.4476 | 18.95 | 17.77 | 213275.1269 | 409812.5600 | 18.85 | 17.68 | 1.0 | 1000.0 | GRP | 2017-11-08 | AMIANTIT | CS | 1 | N | N | B | 1 | 2 | None | None | C2014/14 | Connected / Live | Qatar West | 90 | 90.0 | {99FB5933-6449-4053-9B80-81082146F043} | {"hasZ": true, "rings": [[[213270.1542999996, ... |
Surface_Ground_Water_Culvert.shape
(1208, 80)
Surface_Ground_Water_Culvert_null = pd.DataFrame(Surface_Ground_Water_Culvert.isnull().sum())
Surface_Ground_Water_Culvert_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 814 |
| PROJECT_STATUS | 814 |
| RD_ANAME | 1197 |
| ... | ... |
| TAKAMUL_CATCHMENT | 0 |
| CONFIDENCE | 0 |
| RELIABLE | 23 |
| GLOBALID | 0 |
| SHAPE | 0 |
80 rows × 1 columns
x = Surface_Ground_Water_Culvert.columns
y = Surface_Ground_Water_Culvert.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Culvert", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Culvert Wall feature class has:
• Attribute Fields (Columns): 73
• Total Records (Rows): 606
SG_Water_Culvert_Wall = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGARCLWL")
SG_Water_Culvert_Wall.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | STATUS | SUBTYPE_CODE | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SGW_TYPE | CVT_WALL_NUM | EASTING | NORTHING | GND_LVL | INV_LVL | CVT_SIDE | SURROUND | SURFACE | CONST_DATE | WATER_TABLE | PS_AREA | CVT_SURND_MAT | HEADWALL_LENGTH | HEADWALL_THKNS | SIDEWALL_LENGTH | SIDEWALL_THKNS | REMARKS | PROJ_REF | TAKAMUL_CATCHMENT | CONFIDENCE | RELIABLE | GLOBALID | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 12 | SGARCLWL | IA 12/13 C 063 ST | As Built | None | None | NaN | NaN | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-03-09 13:47:15 | ES/VV | 2020-04-13 | None | None | None | None | ES/BD | 2020-06-11 | None | None | Ashghal | None | CN | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | N | NOH-C4/WW26 | 212962.4165 | 212962.4165 | 20.00 | 18.02 | O | Other | Other | 2017-05-02 | Below Pipe | None | Concrete Line | 1745.0 | 300.0 | 2885.0 | 300.0 | None | C2014/14 | Qatar West | 90 | 90.0 | {8EC1A70B-35CD-4B82-A8B7-53F7E097C758} | {"hasZ": true, "rings": [[[212961.8041000003, ... |
| 1 | 13 | SGARCLWL | IA 12/13 C 063 ST | As Built | None | None | NaN | NaN | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-03-09 13:47:15 | ES/VV | 2020-04-13 | None | None | None | None | ES/BD | 2020-06-11 | None | None | Ashghal | None | CN | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | N | NOH-C4/WW38 | 213371.3940 | 213371.3940 | 21.39 | 19.28 | O | Other | Other | 2017-05-02 | Below Pipe | None | Concrete Line | 1745.0 | 300.0 | 2885.0 | 300.0 | None | C2014/14 | Qatar West | 90 | 90.0 | {5F01D7B0-1E6E-491D-894A-93765BD690B1} | {"hasZ": true, "rings": [[[213373.36349999998,... |
SG_Water_Culvert_Wall.shape
(606, 73)
SG_Water_Culvert_Wall_null = pd.DataFrame(SG_Water_Culvert_Wall.isnull().sum())
SG_Water_Culvert_Wall_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 0 |
| PROJECT_STATUS | 0 |
| RD_ANAME | 580 |
| ... | ... |
| TAKAMUL_CATCHMENT | 0 |
| CONFIDENCE | 0 |
| RELIABLE | 167 |
| GLOBALID | 0 |
| SHAPE | 0 |
73 rows × 1 columns
x = SG_Water_Culvert_Wall.columns
y = SG_Water_Culvert_Wall.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Culvert Wall", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Ditch feature class has:
• Attribute Fields (Columns): 71
• Total Records (Rows): 3,809
SG_Water_Ditch = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGARDITH")
SG_Water_Ditch.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | STATUS | SUBTYPE_CODE | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SGW_TYPE | DITCH_NUM | FM_NODE | FM_EASTING | FM_NORTHING | FM_GND_LVL | TO_NODE | TO_EASTING | TO_NORTHING | TO_GND_LVL | VOLUME | CONST_DATE | DITCH_TYPE | GROUND_TYPE | REMARKS | PS_AREA | PROJ_REF | TAKAMUL_CATCHMENT | CONFIDENCE | RELIABLE | GLOBALID | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 8 | Ditch | IA 12/13 C 063 ST | As Built | None | None | NaN | NaN | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-03-09 13:47:15 | ES/VV | 2020-04-13 | None | None | None | None | ES/BD | 2020-06-17 | None | None | Ashghal | None | CN | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | N | NOH-C4/SWL40/CPT2/PCD1 | Node 1-TOP | 212997.2754 | 417712.2268 | 22.115 | Node 53-BOT | 213005.7090 | 417715.9831 | 21.711 | 1877.36206 | 2016-02-01 | Lined | Fill | None | None | C2014/14 | Qatar West | 90 | NaN | {D86CCE3F-695D-4C39-91D8-3BE36F6341C2} | {"hasZ": true, "rings": [[[213001.7390999999, ... |
| 1 | 9 | Ditch | IA 12/13 C 063 ST | As Built | None | None | NaN | NaN | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-03-09 13:47:15 | ES/VV | 2020-04-13 | None | None | None | None | ES/BD | 2020-06-17 | None | None | Ashghal | None | CN | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | N | NOH-C4/SWL40/CPT1/DCH1 | Node 1-TOP | 213037.1304 | 417709.6751 | 21.768 | Node 54-BOT | 213040.0655 | 417710.7224 | 21.222 | 949.78170 | 2016-02-01 | Lined | Fill | None | None | C2014/14 | Qatar West | 90 | NaN | {AD1D7D21-D856-42D4-8BAD-0BEDCAD7CB16} | {"hasZ": true, "rings": [[[213038.36730000004,... |
SG_Water_Ditch.shape
(3809, 71)
SG_Water_Ditch_null = pd.DataFrame(SG_Water_Ditch.isnull().sum())
SG_Water_Ditch_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 695 |
| PROJECT_STATUS | 695 |
| RD_ANAME | 3685 |
| ... | ... |
| TAKAMUL_CATCHMENT | 6 |
| CONFIDENCE | 0 |
| RELIABLE | 783 |
| GLOBALID | 0 |
| SHAPE | 0 |
71 rows × 1 columns
x = SG_Water_Ditch.columns
y = SG_Water_Ditch.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Ditch", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Micro Tunnel Manhole feature class has:
• Attribute Fields (Columns): 166
• Total Records (Rows): 1,014
Surface_Ground_Water_Micro_Tunnel_Manhole = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGARMTMH")
Surface_Ground_Water_Micro_Tunnel_Manhole.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | ARMTMH_KY | SH_MH_NO | PS_AREA | PROJECT_REF | SURROUND | SURFACE | CONST_DATE | COV_A_EAST | COV_A_NORT | COV_A_LVL | COV_B_EAST | COV_B_NORT | COV_B_LVL | ... | LNDG_SLB_C_SFT_TYPE | WATER_TABLE | TURBULENT | INFILTRATION | STATUS | BENCH_FORM | BENCH_COND | BENCH_PROT | CORNER_A_EAST | CORNER_A_NORT | CORNER_B_EAST | CORNER_B_NORT | CORNER_C_EAST | CORNER_C_NORT | CORNER_D_EAST | CORNER_D_NORT | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 25901 | SGARMTMH | IA 2017C 061G | As Built | None | None | NaN | 91.0 | None | Excel | Manual | None | Drainage O&M | O&M Drainage | 2021-12-14 | ES | 2021-12-14 | None | None | None | NaT | ES | 2021-12-14 | None | None | Ashghal | None | None | SW15/17/1/A10/2 | SW15 | C2018/49 | 1 | 1 | 2020-06-17 | 230471.366 | 375315.057 | 4.305 | NaN | NaN | NaN | ... | None | A | N | N | CN | Y | 1 | 3 | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 1 | {DD1563EC-6A37-48A4-ADC9-61BDBB85479C} | None | None | None | None | 0 | None | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"rings": [[[230470.9249999998, 375316.3499999... |
| 1 | 25898 | SGARMTMH | IA 2017C 062G | As Built | None | None | NaN | 91.0 | None | Excel | Manual | None | Drainage O&M | O&M Drainage | 2021-12-06 | ES | 2021-12-06 | None | None | None | NaT | ES | 2021-12-06 | None | None | Ashghal | None | None | SW15/9/A3/14 | None | C2018/50 | 1 | 1 | 2020-06-17 | 232899.896 | 375216.635 | 5.342 | NaN | NaN | NaN | ... | None | A | N | N | Connected / Live | Y | 1 | 3 | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 1 | {E1ED3F98-1669-46B4-97E2-0C9C96E65C39} | None | None | None | None | 0 | None | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"curveRings": [[[232899.90000000037, 375217.5... |
2 rows × 166 columns
Surface_Ground_Water_Micro_Tunnel_Manhole.shape
(1212, 166)
Surface_Ground_Water_Micro_Tunnel_Manhole_null = pd.DataFrame(Surface_Ground_Water_Micro_Tunnel_Manhole.isnull().sum())
Surface_Ground_Water_Micro_Tunnel_Manhole_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 11 |
| PROJECT_STATUS | 1 |
| RD_ANAME | 1115 |
| ... | ... |
| CONTRACT_ID | 1190 |
| PROJECT_ID | 1212 |
| ASSET_TAG | 1212 |
| ASSET_PRIORITY | 1212 |
| SHAPE | 0 |
166 rows × 1 columns
x = Surface_Ground_Water_Micro_Tunnel_Manhole.columns
y = Surface_Ground_Water_Micro_Tunnel_Manhole.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Micro Tunnel Manhole", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Pond feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Pond = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGARPOND")
SG_Water_Pond.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | STATUS | SUBTYPE_CODE | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SGW_TYPE | POND_NUM | EASTING | NORTHING | HIGH_LVL | LOW_LVL | VOLUME | STRG_CAPC | CONST_DATE | REMARKS | PS_AREA | PROJ_REF | TAKAMUL_CATCHMENT | GLOBALID | CONFIDENCE | RELIABLE | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 12 | Pond | IA 12/13 C 063 ST | As Built | None | None | NaN | NaN | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-03-09 13:47:15 | ES/BD | 2020-04-09 | None | None | None | None | ES/BD | 2020-06-11 | None | None | Ashghal | None | CN | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | N | NOH-C4/PCP14 | 212600.2292 | 412980.7258 | 16.590 | 15.5097 | 2330.0 | 2330.0 | 2017-02-20 | Top of Pond | None | C2014/14 | Qatar West | {AC20647E-8EFD-48F2-BE00-AB0A00634BD4} | 90.0 | 90.0 | {"hasZ": true, "rings": [[[212571.66199999955,... |
| 1 | 13 | Pond | IA 12/13 C 063 ST | As Built | None | None | NaN | NaN | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-03-09 13:47:15 | ES/BD | 2020-04-09 | None | None | None | None | ES/BD | 2020-06-11 | None | None | Ashghal | None | CN | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | N | NOH-C4/PCP6 | 211189.1700 | 404100.9040 | 17.074 | 15.1250 | 1394.0 | 1394.0 | 2017-02-14 | Bottom of Pond | None | C2014/14 | Qatar West | {127B74FC-3BA1-4947-9E42-752DC120A2F1} | 90.0 | 90.0 | {"hasZ": true, "rings": [[[211230.2622999996, ... |
SG_Water_Pond.shape
(1707, 66)
SG_Water_Pond_null = pd.DataFrame(SG_Water_Pond.isnull().sum())
SG_Water_Pond_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 945 |
| PROJECT_STATUS | 945 |
| RD_ANAME | 1600 |
| ... | ... |
| TAKAMUL_CATCHMENT | 0 |
| GLOBALID | 0 |
| CONFIDENCE | 109 |
| RELIABLE | 111 |
| SHAPE | 0 |
66 rows × 1 columns
x = SG_Water_Pond.columns
y = SG_Water_Pond.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Pond", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
SGW Pumping Station Catchment Boundary feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SGW_Pumping_Station = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGARPSCB")
SGW_Pumping_Station.head(2)
| OBJECTID | ID | PSCB | GLOBALID | SUBTYPE_CODE | GFCODE | STATUS | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 10043 | 0.0 | Outfall | {0EACA022-4123-41DA-8ECA-74D24033477F} | 1 | SGARPSCB | Operational | None | None | None | None | 0 | None | None | None | None | None | 51.520798 | 25.345376 | None | None | None | None | None | None | None | None | None | {"rings": [[[230626.86820000038, 397803.989800... |
| 1 | 10036 | NaN | SW29 | {B9270328-026D-4940-B0A4-415709BCC859} | 1 | SGARPSCB | Operational | None | None | None | None | 0 | None | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"rings": [[[231625.7781999996, 393286.1736999... |
SGW_Pumping_Station.shape
(33, 29)
SGW_Pumping_Station_null = pd.DataFrame(SGW_Pumping_Station.isnull().sum())
SGW_Pumping_Station_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| ID | 13 |
| PSCB | 0 |
| GLOBALID | 0 |
| SUBTYPE_CODE | 0 |
| GFCODE | 0 |
| STATUS | 0 |
| MXASSETNUM | 33 |
| MXLOCATION | 33 |
| MXSITEID | 33 |
| MXCREATIONSTATE | 33 |
| ROWSTAMP | 0 |
| MXSTATUS | 33 |
| PARENT_GLOBALID | 33 |
| LOCATION_GLOBAL_ID | 33 |
| DATA_LOAD_ID | 33 |
| DISCIPLINE | 33 |
| GPS_X | 15 |
| GPS_Y | 15 |
| GPS_Z | 33 |
| OPERATOR | 33 |
| MAINTAINER | 33 |
| MUNICIPALITY | 33 |
| DISTRICT | 33 |
| CONTRACT_ID | 33 |
| PROJECT_ID | 33 |
| ASSET_TAG | 33 |
| ASSET_PRIORITY | 33 |
| SHAPE | 0 |
x = SGW_Pumping_Station.columns
y = SGW_Pumping_Station.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("SGW Pumping Station Catchment Boundary", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Swale feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Swale = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGARSWAL")
SG_Water_Swale.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | STATUS | SUBTYPE_CODE | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SGW_TYPE | FM_NODE | FM_EASTING | FM_NORTHING | FM_GND_LVL | FM_BASE_LVL | TO_NODE | TO_EASTING | TO_NORTHING | TO_GND_LVL | TO_BASE_LVL | CONST_DATE | SWALE_TYPE | LINER | BEDDING | BEDDING_THICKNESS | WIDTH | HEIGHT_TR | HEIGHT_OR | WATER_TABLE | GROUND_TYPE | REMARKS | PS_AREA | PROJ_REF | TAKAMUL_CATCHMENT | GLOBALID | CONFIDENCE | RELIABLE | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1067 | Swale | IA 12/13 C 063 ST | As Built | None | None | NaN | NaN | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-03-09 13:47:16 | ES/BD | 2020-04-13 | None | None | None | None | ES/BD | 2020-06-11 | None | None | Ashghal | None | CN | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | N | NOH-C4/SWL40 | 213091.6919 | 418278.9804 | 20.397 | 16.188 | NOH-C4/SWL40-NODE-1 | 213100.2360 | 418282.4567 | 20.397 | 16.188 | 2017-02-01 | Unlined | Y | Other | NaN | 2700.0 | 920 | 620.0 | Below Pipe | Fill | Bottom | None | C2014/14 | Qatar West | {3985C2C2-FE1A-4B6A-BE4D-9A782020A836} | 90 | 90.0 | {"hasZ": true, "rings": [[[213098.98790000007,... |
| 1 | 1068 | Swale | IA 12/13 C 063 ST | As Built | None | None | NaN | NaN | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-03-09 13:47:16 | ES/BD | 2020-04-13 | None | None | None | None | ES/BD | 2020-06-11 | None | None | Ashghal | None | CN | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | N | NOH-C4/SWL41 | 213160.5539 | 419349.0046 | 22.940 | 19.505 | NOH-C4/SWL41-NODE-1 | 213168.7021 | 419350.7597 | 22.940 | 19.505 | 2017-02-01 | Unlined | Y | Other | NaN | 600.0 | 980 | 630.0 | Below Pipe | Fill | Bottom | None | C2014/14 | Qatar West | {E9E910C1-E318-4CEE-AD9C-B6E842AAF53B} | 90 | 90.0 | {"hasZ": true, "rings": [[[213166.84800000023,... |
SG_Water_Swale.shape
(2239, 78)
SG_Water_Swale_null = pd.DataFrame(SG_Water_Swale.isnull().sum())
SG_Water_Swale_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 921 |
| PROJECT_STATUS | 921 |
| RD_ANAME | 2037 |
| ... | ... |
| TAKAMUL_CATCHMENT | 0 |
| GLOBALID | 0 |
| CONFIDENCE | 0 |
| RELIABLE | 701 |
| SHAPE | 0 |
78 rows × 1 columns
x = SG_Water_Pond.columns
y = SG_Water_Pond.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Pond", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Takamul Supply Zone Catchment Boundries feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Takamul = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGARSZCB")
SG_Water_Takamul.head(2)
| OBJECTID | CATCHMENT | CONTRACTOR_ID | CONTRACTOR | GLOBALID | SHAPE | |
|---|---|---|---|---|---|---|
| 0 | 641 | N/A | {6C913EE6-52F7-4E69-A8A2-0A13E5A9DC7A} | {"hasZ": true, "hasM": true, "rings": [[[23654... | ||
| 1 | 642 | Qatar North | C/2020/47 | SUEZ INTERNATIONAL | {4B1FFDA0-39B2-42EC-8534-6186B3C6BFF9} | {"hasZ": true, "hasM": true, "rings": [[[21066... |
SG_Water_Takamul.shape
(4, 6)
SG_Water_Takamul_null = pd.DataFrame(SG_Water_Takamul.isnull().sum())
SG_Water_Takamul_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| CATCHMENT | 0 |
| CONTRACTOR_ID | 0 |
| CONTRACTOR | 0 |
| GLOBALID | 0 |
| SHAPE | 0 |
x = SG_Water_Takamul.columns
y = SG_Water_Takamul.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Takamul Supply Zone Catchment Boundries", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Holding Tanks feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_HTanks = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGARTANK")
SG_Water_HTanks.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | TANK_NUM | TANK_TYPE | PROJECT_REF | CONST_DATE | COV_A_EAST | COV_A_NORT | COV_B_EAST | COV_B_NORT | COV_C_EAST | COV_C_NORT | COV_D_EAST | COV_D_NORT | COV_A_LVL | ... | OT_IL_US_MH_C_INV_LVL | OT_IL_PIPE_C_BD_LVL | OT_IL_PIPE_C_MATERIAL | DESCRIPTION_C | OT_IL_C_EASTING | OT_IL_C_NORTHING | OL_MH_NUM | OL_PIPE_DIA | OL_DEPTH | OL_BDROP | OL_INV_LVL | OL_PIPE_MATERIAL | DESCRIPTION_OL | OL_EASTING | OL_NORTHING | SUBTYPE_CODE | GLOBALID | STATUS | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 15438 | SGARATTK | IA 11 12 C 006 G | As Built | None | None | NaN | 58.0 | None | Excel | Manual | None | Drainage O&M | O&M Drainage | 2017-07-25 09:05:05.000001 | ELANGO | 2017-07-25 09:05:32.000001 | None | None | ELANGO | 2017-07-25 09:05:38.000000 | ELANGO | 2017-07-25 09:05:41 | None | None | Ashghal | None | CM/ATTN1 | 1 | C 2012/53 | 2014-04-01 | 226307.687 | 388052.741 | 226313.417 | 388043.594 | NaN | NaN | NaN | NaN | 13.021 | ... | NaN | None | None | None | NaN | NaN | CM/ATTN1/SWBH1 | 300.0 | 2.26 | None | 10.67 | None | None | 226315.786 | 388042.309 | 1 | {436CB0F5-08F0-4AA0-93A0-A2F43DDAE2ED} | None | 11163835 | None | 101 | NaN | 55002420 | OPERATING | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | NaN | {"rings": [[[226306.85199999996, 388052.932000... |
| 1 | 15435 | SGARLGON | IA 10/11 C 114/G | As Built | None | EAST INDUSTRIAL STREET | NaN | 56.0 | None | Excel | Manual | None | Drainage O&M | O&M Drainage | 2016-03-03 10:42:48.000000 | Elango/AH | 2017-06-18 09:42:57.000000 | None | None | Elango | 2015-10-26 12:19:04.000001 | Elango/AH | 2016-03-03 10:42:58 | None | None | Ashghal | None | 56/EFA | 2 | C 2011/85 | 2014-04-01 | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | ... | NaN | None | None | None | NaN | NaN | None | NaN | NaN | None | NaN | None | None | NaN | NaN | 1 | {222482A2-2FA0-46D2-B6EB-554E19756DE3} | Connected/Live | 10124480 | None | 101 | NaN | 55002416 | OPERATING | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | NaN | {"rings": [[[224316.00100000016, 382121.708000... |
2 rows × 220 columns
SG_Water_HTanks.shape
(405, 220)
SG_Water_HTanks_null = pd.DataFrame(SG_Water_HTanks.isnull().sum())
SG_Water_HTanks_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 21 |
| PROJECT_STATUS | 45 |
| RD_ANAME | 382 |
| ... | ... |
| CONTRACT_ID | 405 |
| PROJECT_ID | 405 |
| ASSET_TAG | 403 |
| ASSET_PRIORITY | 402 |
| SHAPE | 0 |
220 rows × 1 columns
x = SG_Water_HTanks.columns
y = SG_Water_HTanks.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Holding Tanks", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Trench feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Trench = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGARTRCH")
SG_Water_Trench.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | SUBTYPE_CODE | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | ... | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | CALCLEN | SGW_TYPE | TRENCH_NUM | FM_EASTING | FM_NORTHING | FM_GND_LVL | FM_INV_LVL | TO_EASTING | TO_NORTHING | TO_GND_LVL | TO_INV_LVL | DIM_1 | DIM_2 | DIAMETER | MATERIAL | CONST_DATE | PIPE_TYPE | MANUFACTURER | BEDDING | GROUND_TYPE | SLEEVED | INFILTRATION | WATER_TABLE | PIPE_CONDITION | PIPE_CLASS | REMARKS | PS_AREA | PROJ_REF | STATUS | FEATURE_REMARKS | YEAR_LAID | TAKAMUL_CATCHMENT | CONFIDENCE | RELIABLE | GLOBALID | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 333 | SGARTRCH | RTC/041/2012 | As Built Yet to be Approved | None | None | NaN | 51.0 | Excel | Manual | None | Drainage O&M | O&M Drainage | 2020-06-08 11:59:14 | ES | 2020-06-08 11:59:29.000001 | None | NaT | None | NaT | ES | 2020-06-08 11:59:38 | None | None | Ashghal | None | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | ... | None | None | None | None | NaN | P | 51/373/TR1 | 218411.461264 | 395251.604067 | NaN | NaN | NaN | NaN | NaN | NaN | 6.0 | 2.0 | None | None | 2018-11-22 | None | None | GR | 3 | None | Y | B | 1 | None | None | None | M008 | CN | INFILTRATION TRENCH (LxWxD): 6.16m x 1.16m x 1... | None | Qatar North | 45 | NaN | {538E0485-165B-4E5A-AAA9-3C29A9D8CF38} | {"rings": [[[218408.32699999958, 395251.594200... |
| 1 | 334 | SGLNTRCH | M008 | As Built Yet to be Approved | None | None | NaN | NaN | DWG | Digitization | None | Drainage O&M | O&M Drainage | 2020-06-03 00:00:00 | IW | NaT | None | NaT | None | NaT | IW/EA | 2020-06-13 00:00:00 | None | None | Ashghal | None | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | ... | None | None | None | None | NaN | N | 51/2304/TR2 | NaN | NaN | NaN | 9.87 | NaN | NaN | NaN | 9.88 | NaN | NaN | None | None | NaT | None | None | None | None | None | None | None | None | None | None | None | RTC/041/2012 | CN | None | None | Qatar North | 45 | NaN | {FE3CF485-59A1-4745-AED2-896AA76FDF0B} | {"rings": [[[222644.47840000037, 396386.310399... |
2 rows × 84 columns
SG_Water_Trench.shape
(1234, 84)
SG_Water_Trench_null = pd.DataFrame(SG_Water_Trench.isnull().sum())
SG_Water_Trench_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 328 |
| PROJECT_STATUS | 328 |
| RD_ANAME | 1197 |
| ... | ... |
| TAKAMUL_CATCHMENT | 0 |
| CONFIDENCE | 0 |
| RELIABLE | 63 |
| GLOBALID | 0 |
| SHAPE | 0 |
84 rows × 1 columns
x = SG_Water_Trench.columns
y = SG_Water_Trench.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Trench", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Valve Chamber feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Valve_Chamber = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGARVLCH")
SG_Water_Valve_Chamber.head(1)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | ARVLCH_KY | PROJECT_REF | VC_OLD | DEPTH | SURVEY_DATE | PARTITION | SURROUND | SURFACE | FUNCTION | COVER_LVL | COV_SLB_LVL | BASE_SLB_LVL | CONST_DATE | ... | ACC_MATERIAL | CHAM_SHAPE | CHAM_MATERIAL | CHAM_CONST_METHOD | CHAM_COND | CHAM_INT_PROT | CHAM_INT_PROT_COND | CHAM_EXT_PROT | CHAM_EXT_PROT_COND | CHAM_DIM_1 | CHAM_DIM_2 | CHAM_WALL_THKNS | CHAM_HEIGHT | WATER_TABLE | INFILTRATION | STATUS | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 10056 | SGARVLCH | 1A11/12C14G | As Built Yet to be Approved | None | None | NaN | 57.0 | SGARFDCH- Flow Diversion Control Chamber with ... | Excel | Automation | None | Drainage O&M | O&M Drainage | 2019-12-15 | VB | 2019-12-15 11:46:57 | None | None | None | NaT | VB/ES | 2019-12-15 | None | None | Ashghal | None | NaN | C2012/41 | SW10/20/19/3/FC/2 | 3.565 | 2017-04-05 | None | 2 | 3 | 4 | 19.25 | 19.25 | 15.685 | 2017-04-01 | ... | None | 2 | None | None | None | None | None | None | None | 1530.0 | 1900.0 | 300.0 | 3.265 | B | N | CN | 1 | {0F3B5736-02C9-468C-8F00-7570D394EF04} | 13007553 | None | 101 | NaN | 55001847 | OPERATING | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"rings": [[[223755.77529999986, 382096.519999... |
1 rows × 106 columns
SG_Water_Valve_Chamber.shape
(160, 106)
SG_Water_Valve_Chamber_null = pd.DataFrame(SG_Water_Valve_Chamber.isnull().sum())
SG_Water_Valve_Chamber_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 30 |
| PROJECT_STATUS | 36 |
| RD_ANAME | 140 |
| ... | ... |
| CONTRACT_ID | 160 |
| PROJECT_ID | 160 |
| ASSET_TAG | 160 |
| ASSET_PRIORITY | 160 |
| SHAPE | 0 |
106 rows × 1 columns
x = SG_Water_Valve_Chamber.columns
y = SG_Water_Valve_Chamber.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Valve Chamber", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Culvert Pipes feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Culv_Pipes = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGLNCLVT")
SG_Water_Culv_Pipes.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | SUBTYPE_CODE | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | CALCLEN | SGW_TYPE | CVT_NUM | IN_EASTING | IN_NORTHING | IN_GND_LVL | IN_INV_LVL | OUT_EASTING | OUT_NORTHING | OUT_GND_LVL | OUT_INV_LVL | NO_OF_PIPES | DIAMETER | MATERIAL | CONST_DATE | MANUFACTURER | BEDDING | GROUND_TYPE | SLEEVED | INFILTRATION | WATER_TABLE | PIPE_CONDITION | PIPE_CLASS | REMARKS | PS_AREA | PROJ_REF | STATUS | TAKAMUL_CATCHMENT | CONFIDENCE | RELIABLE | GLOBALID | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 14 | SGLNCLVT | IA 12/13 C 054 ST | As Built | None | None | NaN | 91.0 | Excel | Manual | None | Drainage O&M | O&M Drainage | 2020-04-07 14:31:27 | ES | 2020-04-07 14:31:32 | None | None | None | None | ES | 2020-04-07 14:31:40 | None | None | Ashghal | None | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | 55.917517 | N | IC32/MP/CV/03 | 220047.608 | 376598.733 | 30.476 | 29.676 | 220072.0566 | 376548.5627 | 29.878 | 29.078 | NaN | 600.0 | RC | 2014-12-23 | DOHA PRECAST FACTORY | GR | 1 | N | N | B | OK | 0 | None | None | C2013/56 | CN | Qatar West | 75 | NaN | {65BA2311-E307-4934-9DD6-204E6048491D} | {"paths": [[[220047.608, 376598.7329999991], [... |
| 1 | 15 | SGLNCLVT | IA 12/13 C 054 ST | As Built | None | None | NaN | 81.0 | Excel | Manual | None | Drainage O&M | O&M Drainage | 2020-04-07 14:31:27 | ES | 2020-04-07 14:31:32 | None | None | None | None | ES | 2020-04-07 14:31:40 | None | None | Ashghal | None | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | 18.421205 | N | IC32/RP/CV/14 | 220025.562 | 376651.387 | 30.920 | 30.120 | 220017.2400 | 376667.3850 | 30.850 | 30.050 | NaN | 600.0 | RC | 2015-06-16 | DOHA PRECAST FACTORY | GR | 1 | N | N | B | OK | 0 | None | None | C2013/56 | CN | Qatar West | 75 | NaN | {53F4EB63-317C-428B-BF70-4141B5F90AA2} | {"paths": [[[220025.56199999992, 376651.387000... |
SG_Water_Culv_Pipes.shape
(580, 80)
SG_Water_Culv_Pipes_null = pd.DataFrame(SG_Water_Culv_Pipes.isnull().sum())
SG_Water_Culv_Pipes_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 58 |
| PROJECT_STATUS | 58 |
| RD_ANAME | 580 |
| ... | ... |
| TAKAMUL_CATCHMENT | 0 |
| CONFIDENCE | 0 |
| RELIABLE | 226 |
| GLOBALID | 0 |
| SHAPE | 0 |
80 rows × 1 columns
x = SG_Water_Culv_Pipes.columns
y = SG_Water_Culv_Pipes.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Culvert Pipes", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Duct feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Duct = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGLNDUCT")
SG_Water_Duct.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | SUBTYPE_CODE | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | ... | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | CALCLEN | SGW_TYPE | DUCT_NUM | IN_EASTING | IN_NORTHING | IN_GND_LVL | IN_INV_LVL | IN_ENDCAP | OUT_EASTING | OUT_NORTHING | OUT_GND_LVL | OUT_INV_LVL | OUT_ENDCAP | NO_OF_PIPES | DIAMETER | MATERIAL | CONST_DATE | MANUFACTURER | BEDDING | GROUND_TYPE | SLEEVED | INFILTRATION | WATER_TABLE | PIPE_CONDITION | PIPE_CLASS | REMARKS | STATUS | PS_AREA | PROJ_REF | TAKAMUL_CATCHMENT | CONFIDENCE | RELIABLE | GLOBALID | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 321 | SGLNDUCT | S722 | As Built | None | None | None | None | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-09-26 | BH | None | None | None | None | None | BH/EA | 2020-10-03 | None | None | Ashghal | None | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | ... | None | None | None | None | None | None | 8.672706 | P | 61/833/01/SD | 232001.574 | 397265.831 | 2.44 | 1.28 | N | 231992.903 | 397265.659 | 2.56 | 1.27 | N | 1 | 200 | GP | 2019-01-05 | AMIANTIT QATAR PIPES | GR | 2 | N | N | B | 1 | 1 | None | CN | None | S722 | Qatar North | 95 | None | {D3C7FDF0-CD38-4D66-BD80-94C3B5992704} | {"paths": [[[232001.57400000002, 397265.831000... |
1 rows × 82 columns
SG_Water_Duct.shape
(1, 82)
SG_Water_Duct_null = pd.DataFrame(SG_Water_Duct.isnull().sum())
SG_Water_Duct_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 0 |
| PROJECT_STATUS | 0 |
| RD_ANAME | 1 |
| ... | ... |
| TAKAMUL_CATCHMENT | 0 |
| CONFIDENCE | 0 |
| RELIABLE | 1 |
| GLOBALID | 0 |
| SHAPE | 0 |
82 rows × 1 columns
x = SG_Water_Duct.columns
y = SG_Water_Duct.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Duct", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Gully Connection feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Gully_Connec = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGLNGCON")
SG_Water_Gully_Connec.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | PROJECT_REF | GULLY_NO | COVER_LVL | BD_INV_LVL | CON_INV_LVL | DIAMETER | MATERIAL | MANUFACTURER | BEDDING | GROUND_TYPE | CONNECTED_TO | STATUS | CONNECTION_TYPE | CONST_DATE | DIST_DSMH_GULLY | LNGCON_KY | CONFIDENCE | RELIABLE | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 3975926 | SGLNGCON | DSC0074C1 | None | 0.0 | 0.0 | None | None | NaT | PC | 2016-01-27 | None | None | None | NaT | NaT | None | None | None | C693 | NaN | NaN | NaN | 150.0 | UP | GOODWIN QATAR | GRANULAR | ROCK | MANHOLE | CONNECTED | STRAIGHT | None | NaN | 0.0 | 0.0 | 0.0 | 1 | {B18843AA-4FF9-45BB-9EBB-3F092F121ECF} | 426336 | None | 101 | None | 55096621 | OPERATING | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"paths": [[[223506.31300000008, 387157.484099... | |||||||||
| 1 | 3974834 | SGLNGCON | None | 0.0 | 0.0 | None | None | NaT | NaT | None | None | None | NaT | NaT | None | None | None | NaN | NaN | NaN | 0.0 | None | NaN | 0.0 | 95.0 | 800.0 | 1 | {C72DFDBD-5667-46C3-B5E3-ADD64D05EAFA} | 424420 | None | 101 | None | 55095454 | OPERATING | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"paths": [[[229550.72300000023, 391291.446000... |
SG_Water_Gully_Connec.shape
(159480, 69)
SG_Water_Gully_Connec_null = pd.DataFrame(SG_Water_Gully_Connec.isnull().sum())
SG_Water_Gully_Connec_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 10649 |
| PROJECT_STATUS | 11903 |
| RD_ANAME | 150607 |
| ... | ... |
| CONTRACT_ID | 156164 |
| PROJECT_ID | 159480 |
| ASSET_TAG | 159480 |
| ASSET_PRIORITY | 159480 |
| SHAPE | 0 |
69 rows × 1 columns
x = SG_Water_Gully_Connec.columns
y = SG_Water_Gully_Connec.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Gully Connection", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Gully Connection Stub Pipe feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_GullyCon_Stub_Pipe = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGLNGCST")
SG_Water_GullyCon_Stub_Pipe.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | PROJECT_REF | GULLY_NO | COVER_LVL | BD_INV_LVL | CON_INV_LVL | DIAMETER | MATERIAL | MANUFACTURER | BEDDING | GROUND_TYPE | CONNECTED_TO | STATUS | CONNECTION_TYPE | CONST_DATE | DIST_DSMH_GULLY | LNGCON_KY | CONFIDENCE | RELIABLE | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 21506 | SGLNGCST | CP05B | As Built | None | None | None | 69.0 | LIGHT RAIL TRANSIT [LRT] FEUTURE DRAINAGE CONN... | Excel | Manual | None | Drainage O&M | O&M Drainage | 2018-09-16 | ES | 2018-09-16 | None | None | None | None | ES | 2018-09-16 | None | None | Ashghal | None | 2888 | SD1/2/LRT | 4.214 | None | 4.744 | 200 | VC | SVCP | GR | 1 | 2 | NC | S | 2013-03-16 | NaN | None | NaN | NaN | 1 | {FDF8599B-3C5B-4832-BA3E-176C8C5522BE} | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"paths": [[[227806.09100000001, 408816.958000... |
| 1 | 21503 | SGLNGCST | CP05B | As Built | None | None | None | 69.0 | LIGHT RAIL TRANSIT [LRT] FEUTURE DRAINAGE CONN... | Excel | Manual | None | Drainage O&M | O&M Drainage | 2018-09-16 | ES | 2018-09-16 | None | None | None | None | ES | 2018-09-16 | None | None | Ashghal | None | 2888 | SD1/7/LRT | 6.049 | None | 6.049 | 200 | VC | SVCP | GR | 1 | 2 | NC | S | 2015-08-25 | NaN | None | NaN | NaN | 1 | {631658E1-C8E0-4D97-8370-5D092F591C5E} | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"paths": [[[227619.7259999998, 408773.3049999... |
SG_Water_GullyCon_Stub_Pipe.shape
(529, 69)
SG_Water_GullyCon_Stub_Pipe_null = pd.DataFrame(SG_Water_GullyCon_Stub_Pipe.isnull().sum())
SG_Water_GullyCon_Stub_Pipe_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 0 |
| PROJECT_STATUS | 0 |
| RD_ANAME | 529 |
| ... | ... |
| CONTRACT_ID | 529 |
| PROJECT_ID | 529 |
| ASSET_TAG | 529 |
| ASSET_PRIORITY | 529 |
| SHAPE | 0 |
69 rows × 1 columns
x = SG_Water_GullyCon_Stub_Pipe.columns
y = SG_Water_GullyCon_Stub_Pipe.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Gully Connection Stub Pipe", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Micro Tunnel Sewer Pipe feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_MT_Sewer_Pipe = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGLNMTSW")
SG_Water_MT_Sewer_Pipe.head(1)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | PROJECT_REF | PIPE_TYPE | PS_AREA | FROM_MH | TO_MH | FM_SEW_INV_LVL | TO_SEW_INV_LVL | FM_INV_LVL | TO_INV_LVL | BD_INV_LVL | DIAMETER | STATUS | MATERIAL | ... | LIMIT_DCH | PIPE_LAID_DATE | TEST_DATE | TEST_TYPE | TEST_PRSR | DROP_IN_PRESSURE | ALLOW_DROP | CCTV_SURVEY | DURATION_HRS | DURATION_MIN | RESULT | ORFC_INV_LVL | ORIFICE_DIA | ORFC_DCH_COEF | S_ORFC_DCH_COED | INFILTRATION | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 85351 | SGLNMTSW | IA 2017 C022 G | As Built Yet to be Approved | None | None | NaN | NaN | None | Excel | Automation | None | Drainage O&M | O&M Drainage | 2021-05-27 | BD | NaT | None | None | None | NaT | 27-May-21 | 2021-05-27 | None | None | Ashghal | None | C2018/69 | 1 | SW10 | SW10/29/43/4 | SW10/29/43/3 | 24.33 | 24.16 | NaN | NaN | NaN | 1400.0 | CN | RC | ... | None | 2019-01-27 | 2020-11-11 | V | NaN | NaN | NaN | N | None | None | P | NaN | NaN | NaN | None | N | 1 | {63E34D08-8C40-482F-A26A-55D1AB1D4996} | 15709987 | None | 101 | None | 62682488 | OPERATING | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"paths": [[[217610.3389999997, 387637.6190000... |
1 rows × 91 columns
SG_Water_MT_Sewer_Pipe.shape
(4547, 91)
SG_Water_MT_Sewer_Pipe_null = pd.DataFrame(SG_Water_MT_Sewer_Pipe.isnull().sum())
SG_Water_MT_Sewer_Pipe_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 12 |
| PROJECT_STATUS | 9 |
| RD_ANAME | 1774 |
| ... | ... |
| CONTRACT_ID | 4526 |
| PROJECT_ID | 4547 |
| ASSET_TAG | 4547 |
| ASSET_PRIORITY | 4547 |
| SHAPE | 0 |
91 rows × 1 columns
x = SG_Water_MT_Sewer_Pipe.columns
y = SG_Water_MT_Sewer_Pipe.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Micro Tunnel Sewer Pipe", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Rising Main feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Rising_Main = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGLNRSMN")
SG_Water_Rising_Main.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | PROJECT_REF | PS_AREA | FM_GND_LVL | TO_GND_LVL | FM_INV_LVL | TO_INV_LVL | DIAMETER | MATERIAL | MANUFACTURER | JOINT_TYPE | GROUND_TYPE | BEDDING | WATER_TABLE | ... | YEAR_LAID | TEST_DATE | TEST_TYPE | PRESSURE | DROP_IN_PRESSURE | ALLOW_DROP | DURATION_HRS | DURATION_MIN | RESULT | MAX_FLOW | LNRSMN_KY | CONFIDENCE | RELIABLE | FM_PS | TO_PS | TO_STW | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 46658 | SGLNRSMN | ERC1400/C4 | None | None | None | NaN | NaN | None | None | None | None | None | 2013-01-10 | None | NaT | None | NaT | None | NaT | None | NaT | None | None | Ashghal | None | 1400/C4 | None | 9.42 | 9.55 | 7.2 | 6.804 | 600.0 | DI | SADIP KSA | FL | 1 | GR | B | ... | 2006.0 | NaT | None | NaN | NaN | NaN | NaN | NaN | None | 0.0 | 0.0 | 0.0 | 0.0 | None | None | None | 1 | {DE35389E-5969-48E5-AA9D-C5C2D4D200AB} | 585384 | None | 101 | None | 39472076 | OPERATING | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"paths": [[[235558.80939999968, 388052.795099... | |
| 1 | 46660 | SGLNRSMN | DSC0021 | None | None | None | NaN | NaN | None | None | None | None | None | 2003-01-18 | EJRC | 2015-02-11 14:30:16 | None | NaT | None | NaT | None | NaT | None | None | Ashghal | None | C391 | None | 0.00 | 0.00 | 0.0 | 0.000 | 700.0 | DI | ... | NaN | NaT | None | NaN | NaN | NaN | NaN | NaN | None | 0.0 | 293.0 | 0.0 | 0.0 | None | None | None | 2 | {937E4FE1-AB5B-4489-9ADF-B9774FBB0EE7} | 585622 | None | 101 | None | 39472078 | DECOMM | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"paths": [[[227031.375, 389383.25], [226898.2... |
2 rows × 86 columns
SG_Water_Rising_Main.shape
(2211, 86)
SG_Water_Rising_Main_null = pd.DataFrame(SG_Water_Rising_Main.isnull().sum())
SG_Water_Rising_Main_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 22 |
| PROJECT_STATUS | 283 |
| RD_ANAME | 1880 |
| ... | ... |
| CONTRACT_ID | 2211 |
| PROJECT_ID | 2211 |
| ASSET_TAG | 2211 |
| ASSET_PRIORITY | 2211 |
| SHAPE | 0 |
86 rows × 1 columns
x = SG_Water_Rising_Main.columns
y = SG_Water_Rising_Main.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Rising Main", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Sewer feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Sewer = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGLNSEWR")
SG_Water_Sewer.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | CORROSION_PROT | CORROSION_PROT_MATERIAL | PIPE_INST_METHOD | SEW_FUNCTION | PS_AREA | PROJECT_REF | LAND_PIPE_TYPE | MAIN_PIPE_TYPE | FROM_MH | FM_INV_LVL | TO_MH | TO_INV_LVL | BD_INV_LVL | ... | MAIN_FLOW | YEAR_LAID | TEST_DATE | TEST_TYPE | PRESSURE | DROP_IN_PRESSURE | ALLOW_DROP | CCTV_SURVEY | DURATION_MIN | DURATION_HRS | RESULT | CONFIDENCE | RELIABLE | FM_PS | TO_PS | LNSEWR_KY | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1706561 | MAIN | IA 13/14 C 67 ST | As Built | None | None | NaN | 57.0 | As per SR#10184771,Existing sewer pipe CCTV in... | Excel | Automation | None | Drainage O&M | O&M Drainage | 2019-11-28 | IW | 2022-01-20 | None | None | None | NaT | IW/EA | 2019-12-14 16:49:19 | None | None | Ashghal | None | N | None | 1 | 2 | None | C2015/84 | None | 1 | SW16/11/26/3 | 18.490 | SW16/11/26/2 | 18.250 | NaN | ... | NaN | 12/06/2018 | 2018-06-12 | V | NaN | NaN | NaN | N | NaN | NaN | P | 45.0 | 45.0 | None | None | NaN | 1 | {02F45E21-B0CD-47A3-8332-FF0754A6226A} | 13006628 | None | 101 | None | 69720462 | OPERATING | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"paths": [[[222539.33600000013, 378466.846000... |
| 1 | 1706243 | LAND | IA 2017C 063G | As Built | None | None | NaN | 91.0 | None | Excel | Manual | None | Drainage O&M | O&M Drainage | 2021-12-22 | ES | 2021-12-22 | None | None | None | NaT | ES | 2021-12-22 00:00:00 | None | None | Ashghal | None | N | None | 1 | 2 | None | C2018/51 | 2 | None | 91/1155/MH9 | 7.178 | 91/1155/MH8 | 7.302 | NaN | ... | NaN | 2019 | NaT | None | NaN | NaN | NaN | None | NaN | NaN | None | 100.0 | 100.0 | None | None | NaN | 1 | {2CBB017B-F45A-43E7-BA8E-550102FBC8BC} | 16523914 | None | 101 | None | 68666323 | NOT READY | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"paths": [[[235069.5120000001, 374537.0109999... |
2 rows × 105 columns
SG_Water_Sewer.shape
(51049, 105)
SG_Water_Sewer_null = pd.DataFrame(SG_Water_Sewer.isnull().sum())
SG_Water_Sewer_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 497 |
| PROJECT_CODE | 1118 |
| PROJECT_STATUS | 1112 |
| RD_ANAME | 48553 |
| ... | ... |
| CONTRACT_ID | 50325 |
| PROJECT_ID | 51049 |
| ASSET_TAG | 51049 |
| ASSET_PRIORITY | 51049 |
| SHAPE | 0 |
105 rows × 1 columns
x = SG_Water_Sewer.columns
y = SG_Water_Sewer.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Sewer", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Stub Pipe feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Stub_Pipe = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGLNSTUB")
SG_Water_Stub_Pipe.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | PS_AREA | PROJECT_REF | LAND_PIPE_TYPE | MAIN_PIPE_TYPE | FROM_MH | FM_INV_LVL | TO_MH | TO_INV_LVL | BD_INV_LVL | LAND_FM_SEW_INV_LVL | MAIN_FM_SEW_INV_LVL | TO_LAND_SEW_INV_LVL | TO_MAIN_SEW_INV_LVL | ... | MAIN_FLOW | YEAR_LAID | TEST_DATE | TEST_TYPE | PRESSURE | DROP_IN_PRESSURE | ALLOW_DROP | CCTV_SURVEY | DURATION_MIN | DURATION_HRS | RESULT | CONFIDENCE | RELIABLE | FM_PS | TO_PS | LNSTUB_KY | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 16191 | SGLNSTUB | IA 13/14/C 068 ST | As Built | None | None | NaN | 57.0 | None | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-04-25 | VV | 2021-06-14 | None | None | None | NaT | EA/BD | 2021-11-28 | None | None | Ashghal | None | None | C2015/73 | None | 1 | 57/EFL1/1/6/24/9/ST1 | 21.995 | 57/EFL1/1/6/24/9 | 21.70 | NaN | NaN | 21.70 | NaN | 21.70 | ... | NaN | 2018 | 2018-08-30 | V | NaN | NaN | NaN | N | NaN | NaN | P | 95.0 | 95.0 | None | None | None | 1 | {A34DBA7C-3E59-428E-A053-598577C7EF85} | None | None | None | None | 0 | None | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"paths": [[[222230.4709999999, 379698.6500000... |
| 1 | 16188 | SGLNSTUB | QS049 | As Built | None | None | NaN | NaN | SI-01 | Excel | Automation | None | Drainage O&M | O&M Drainage | 2021-08-25 | VB | NaT | None | None | None | NaT | BD | 2021-09-28 | None | None | Ashghal | None | None | C2019/29 | None | 1 | 91/922/MH1 | -1.985 | Node 1 | -2.21 | NaN | NaN | -2.21 | NaN | -2.21 | ... | NaN | 2019 | 1905-07-11 | W | NaN | NaN | NaN | Y | NaN | NaN | P | 95.0 | 95.0 | None | None | None | 1 | {3D69C7B5-A805-439F-B63F-313B5948D155} | None | None | None | None | 0 | None | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"paths": [[[233019.45199999958, 377921.785499... |
2 rows × 101 columns
SG_Water_Stub_Pipe.shape
(576, 101)
SG_Water_Stub_Pipe_null = pd.DataFrame(SG_Water_Stub_Pipe.isnull().sum())
SG_Water_Stub_Pipe_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 0 |
| PROJECT_STATUS | 0 |
| RD_ANAME | 540 |
| ... | ... |
| CONTRACT_ID | 575 |
| PROJECT_ID | 576 |
| ASSET_TAG | 576 |
| ASSET_PRIORITY | 576 |
| SHAPE | 0 |
101 rows × 1 columns
x = SG_Water_Stub_Pipe.columns
y = SG_Water_Stub_Pipe.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Stub Pipe", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
SGW Sewer Catchment Boundary feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SGW_Sewer_Catchment_Boundary = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGLNSWCB")
SGW_Sewer_Catchment_Boundary.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | CORROSION_PROT | CORROSION_PROT_MATERIAL | PIPE_INST_METHOD | SEW_FUNCTION | PS_AREA | PROJECT_REF | LAND_PIPE_TYPE | MAIN_PIPE_TYPE | FROM_MH | FM_INV_LVL | TO_MH | TO_INV_LVL | BD_INV_LVL | LAND_FM_SEW_INV_LVL | MAIN_FM_SEW_INV_LVL | TO_LAND_SEW_INV_LVL | TO_MAIN_SEW_INV_LVL | LAND_PIPE_DIAMETER | MAIN_PIPE_DIAMETER | LAND_PIPE_STATUS | MAIN_PIPE_STATUS | LAND_PIPE_MATERIAL | MAIN_PIPE_MATERIAL | LAND_PIPE_MANUFACTURER | MAIN_PIPE_MANUFACTURER | LAND_GROUND_TYPE | MAIN_GROUND_TYPE | LAND_PIPE_BEDDING | MAIN_PIPE_BEDDING | LAND_INFILTRATION | MAIN_INFILTRATION | LAND_WATER_TABLE | MAIN_WATER_TABLE | LAND_PIPE_CONDITION | MAIN_PIPE_CONDITION | LAND_PIPE_CLASS | MAIN_PIPE_CLASS | ... | ALLOW_DROP | CCTV_SURVEY | DURATION_MIN | DURATION_HRS | RESULT | CONFIDENCE | RELIABLE | FM_PS | TO_PS | LNSEWR_KY | SUBTYPE_CODE | GLOBALID | FEATURE_REMARKS | DATE_CREATED | EDITED_BY | SGSEW_ID | ENABLED | CATCHMENT2 | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 389268 | MAIN | DSC0042 | None | None | None | None | SW5 | None | SW5/6/25/7/1 | 5.68 | SW5/6/25/7 | 3.85 | 5.31 | 0.0 | 0.0 | None | None | 0 | 300 | CN | VC | SAUDI VCP | SAUDI VCP | ROCK | GRANULAR IN FILTER MEMBRANE | EXTRA STRENGTH | ... | 0 | 0 | 0 | None | None | 3363 | 1 | {D62C357E-7BEA-4D45-9273-A3F0AEEC0601} | None | NaT | 8845 | 1 | PSG-SW5 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"paths": [[[228518.1129999999, 386628.6779999... | ||||||||||||||||||
| 1 | 389265 | MAIN | ERC1400_D20 | None | None | None | None | ERC1400_D20 | None | 0.00 | 0.00 | 0.00 | 0.0 | 0.0 | None | None | 0 | 400 | VC | ... | 0 | 0 | 0 | None | None | 0 | 1 | {21D415D9-5D94-4AE6-9048-0569C7E87B31} | None | 2013-12-05 | Lateef | 6805 | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"paths": [[[229002.85979999974, 389267.411800... |
2 rows × 87 columns
SGW_Sewer_Catchment_Boundary.shape
(10661, 87)
SGW_Sewer_Catchment_Boundary_null = pd.DataFrame(SGW_Sewer_Catchment_Boundary.isnull().sum())
SGW_Sewer_Catchment_Boundary_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 0 |
| CORROSION_PROT | 10661 |
| CORROSION_PROT_MATERIAL | 10661 |
| ... | ... |
| CONTRACT_ID | 10661 |
| PROJECT_ID | 10661 |
| ASSET_TAG | 10661 |
| ASSET_PRIORITY | 10661 |
| SHAPE | 1 |
87 rows × 1 columns
x = SGW_Sewer_Catchment_Boundary.columns
y = SGW_Sewer_Catchment_Boundary.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("SGW Sewer Catchment Boundary", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Trench feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Trench = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGLNTRCH")
SG_Water_Trench.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | SUBTYPE_CODE | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | ... | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | CALCLEN | SGW_TYPE | TRENCH_NUM | FM_EASTING | FM_NORTHING | FM_GND_LVL | FM_INV_LVL | TO_EASTING | TO_NORTHING | TO_GND_LVL | TO_INV_LVL | DIM_1 | DIM_2 | DIAMETER | MATERIAL | CONST_DATE | PIPE_TYPE | MANUFACTURER | BEDDING | GROUND_TYPE | SLEEVED | INFILTRATION | WATER_TABLE | PIPE_CONDITION | PIPE_CLASS | REMARKS | PS_AREA | PROJ_REF | STATUS | FEATURE_REMARKS | YEAR_LAID | TAKAMUL_CATCHMENT | CONFIDENCE | RELIABLE | GLOBALID | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 14 | SGLNTRCH | IA 12/13 C 054 ST | As Built | None | None | NaN | 57.0 | Excel | Manual | None | Drainage O&M | O&M Drainage | 2020-04-07 12:00:44 | ES | 2020-04-07 12:00:50 | None | None | None | None | ES | 2020-04-07 12:01:31 | None | None | Ashghal | None | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | ... | None | None | None | None | 129.999252 | N | IC31/SAK/TR/01 | 222676.308 | 378611.618 | 22.80 | 21.230 | 222605.340 | 378720.537 | 23.310 | 21.290 | 3000.0 | 2000.0 | 300 | UP | 2017-11-25 | 1 | HEPWORTH | GR | 1 | N | N | B | 1 | None | None | None | C2013/56 | CN | None | 5/27/2015 | Qatar West | 75 | NaN | {08A2547C-F117-4965-A569-C1DA40078222} | {"paths": [[[222676.3080000002, 378611.6180000... |
| 1 | 16 | SGLNCHNL | IA 12/13 C 054 ST | As Built | None | None | NaN | 91.0 | Excel | Manual | None | Drainage O&M | O&M Drainage | 2020-04-07 12:00:44 | ES | 2020-04-07 12:00:50 | None | None | None | None | ES | 2020-04-07 12:01:31 | None | None | Ashghal | None | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | ... | None | None | None | None | 140.738211 | N | IC31/CIU/13 | 223264.450 | 377628.382 | 20.54 | 21.766 | 223332.009 | 377504.930 | 20.865 | 20.966 | 400.0 | 645.0 | None | VC | 2016-02-03 | 1 | ACO SYSTEMS FZE | CS | 3 | N | N | B | 1 | None | None | None | C2013/56 | CN | None | 5/27/2015 | Qatar West | 75 | NaN | {AE222660-DE7F-45A5-97BC-565AE2A59B1E} | {"paths": [[[223264.45010000002, 377628.382099... |
2 rows × 84 columns
SG_Water_Trench.shape
(1725, 84)
SG_Water_Trench_null = pd.DataFrame(SG_Water_Trench.isnull().sum())
SG_Water_Trench_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 328 |
| PROJECT_STATUS | 328 |
| RD_ANAME | 1722 |
| ... | ... |
| TAKAMUL_CATCHMENT | 0 |
| CONFIDENCE | 0 |
| RELIABLE | 1099 |
| GLOBALID | 0 |
| SHAPE | 0 |
84 rows × 1 columns
x = SG_Water_Trench.columns
y = SG_Water_Trench.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Trench", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Borehole feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
Borehole = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDBRHL")
Borehole.head(2)
| OBJECTID | AREA_ | PERIMETER | SGNDBRHL_ | SGNDBRHL_I | BH_ | BH_ID | GFCODE | CONFIDENCE | DB_DATE | RELIABLE | POS_SOURCE | NDBRHL_KY | OLDREF | Z | DT | X_COORD | Y_COORD | BH_OLD | GROUND_LVL | WELL_POINT | SURVEY | PUMP_RATE | DEPTH | USE | OWNER | DRILL_YEAR | DRILL_BY | AQUIFER | REMARKS | WT_82 | WT_83 | WT_84 | WT_85 | WT_86 | WT_87 | WT_88 | WT_89 | WT_90 | WT_91 | WT_92 | WT_93 | GLOBALID | CREATED_USER | CREATED_DATE | LAST_EDITED_USER | LAST_EDITED_DATE | SUBTYPE_CODE | STATUS | STREET_NO | ZONE_NO | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 10512 | 0.0 | 0.0 | 27.0 | 27.0 | 27.0 | 27.0 | SGNDBRHL | 0.0 | NaT | 0.0 | 0.0 | 2239/91 | 6.633 | 1993-09-22 | 224561.899 | 394347.850 | 2239/91 | 6.58 | top of concrete plinth (0.08 m) | 6.66 | 0.0 | 4.2 | old production well, monitor | 0.0 | upper dammam/rus) | 2.40 | 0.00 | 0.00 | 0.00 | 0.0 | 0.0 | 0.3 | 0.60 | 0.45 | 1.58 | 2.20 | 2.35 | {9412DB48-E570-474C-AE92-B0BB7AD9B35E} | None | NaT | None | NaT | 1 | Operational | NaN | NaN | 585665 | None | 101 | None | 55002102 | OPERATING | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"x": 224561.90629999992, "y": 394347.84380000... | ||||
| 1 | 10547 | 0.0 | 0.0 | 3.0 | 3.0 | 3.0 | 3.0 | SGNDBRHL | 0.0 | NaT | 0.0 | 0.0 | 2239/46 | 9.699 | 1993-09-12 | 224911.025 | 396150.579 | 2239/46 | 9.62 | top of plinth | 9.72 | 0.0 | 60.0 | grass watering | 0.0 | upper dammam / rus | As per SR#10173399 borehole are Abandoned | 7.28 | 6.07 | 5.68 | 4.82 | 3.8 | 4.2 | 3.2 | 3.37 | 3.00 | 3.53 | 3.88 | 3.67 | {1EC12BF8-1713-4CAD-9503-8C99E0E3412D} | None | NaT | BD | 2021-07-05 | 2 | AB | NaN | NaN | 585641 | None | 101 | None | 63976074 | DECOMM | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"x": 224911.03129999992, "y": 396150.59380000... |
Borehole.shape
(119, 73)
Borehole_null = pd.DataFrame(Borehole.isnull().sum())
Borehole_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| AREA_ | 49 |
| PERIMETER | 49 |
| SGNDBRHL_ | 49 |
| SGNDBRHL_I | 49 |
| ... | ... |
| CONTRACT_ID | 119 |
| PROJECT_ID | 119 |
| ASSET_TAG | 119 |
| ASSET_PRIORITY | 119 |
| SHAPE | 0 |
73 rows × 1 columns
x = Borehole.columns
y = Borehole.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Borehole", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Catchbasin feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Catchbasin = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDCBIN")
SG_Water_Catchbasin.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | NDMNHL_KY | EASTING | NORTHING | STATUS | SUBTYPE_CODE | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | ... | CB_NUM | COV_A_EAST | COV_A_NORT | COV_A_LVL | COV_B_EAST | COV_B_NORT | COV_B_LVL | PIPE_INV_LVL | BENCH_INV_LVL | DEPTH | CB_TYPE | SURROUND | SURFACE | CONST_DATE | WATER_TABLE | TURBULENT | INFILTRATION | BENCH_FORM | BENCH_COND | BENCH_PROT | PS_AREA | CHAM_SHAPE | CHAM_MATERIAL | CHAM_CONST_METHOD | CHAM_COND | CHAM_INT_PROT | CHAM_INT_PROT_COND | CHAM_EXT_PROT | CHAM_EXT_PROT_COND | CHAM_WALL_THKNS | CHAM_DIM_1 | CHAM_DIM_2 | CHAM_HEIGHT | REMARKS | PROJ_REF | TAKAMUL_CATCHMENT | CONFIDENCE | RELIABLE | GLOBALID | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 321 | SGNDCBIN | IA 2016 C 019I | As Built | None | None | None | None | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-10-19 | BH | None | None | None | None | None | BD | 2020-10-29 | None | None | Ashghal | None | None | 212943.767 | 407212.687 | CN | 1 | None | None | None | None | 0 | None | None | None | None | ... | 71/1200/DCH4/CP1 | None | None | None | 212943.767 | 407212.687 | 20.316 | 19.629 | 18.959 | 0.687 | N | Central Median | Natural | 2018-03-11 | A | Y | Y | Y | OK | Bitumen Paint | None | Rectangular | Concrete | Cast In Situ | OK | Bitumen Paint | OK | Bitumen Paint | OK | 225 | 1500 | 800 | 2.066 | None | C2017/78 | Qatar North | 95 | 95 | {C642AA3D-2D38-4622-8997-88FA0E1A62AB} | {"x": 212943.767, "y": 407212.68700000085, "sp... |
| 1 | 322 | SGNDCBIN | IA 2016 C 019I | As Built | None | None | None | None | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-10-19 | BH | None | None | None | None | None | BD | 2020-10-29 | None | None | Ashghal | None | None | 213555.756 | 407417.412 | CN | 1 | None | None | None | None | 0 | None | None | None | None | ... | 71/1200/DCH4/CP2 | None | None | None | 213555.756 | 407417.412 | 17.530 | 16.640 | 15.970 | 0.890 | N | Central Median | Natural | 2018-03-11 | A | Y | Y | Y | OK | Bitumen Paint | None | Rectangular | Concrete | Cast In Situ | OK | Bitumen Paint | OK | Bitumen Paint | OK | 225 | 1500 | 800 | 2.097 | None | C2017/78 | Qatar North | 95 | 95 | {730E84A8-73BF-4101-A336-33093EA1D432} | {"x": 213555.75600000005, "y": 407417.41200000... |
2 rows × 93 columns
SG_Water_Catchbasin.shape
(4, 93)
SG_Water_Catchbasin_null = pd.DataFrame(SG_Water_Catchbasin.isnull().sum())
SG_Water_Catchbasin_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 0 |
| PROJECT_STATUS | 0 |
| RD_ANAME | 4 |
| ... | ... |
| TAKAMUL_CATCHMENT | 0 |
| CONFIDENCE | 0 |
| RELIABLE | 0 |
| GLOBALID | 0 |
| SHAPE | 0 |
93 rows × 1 columns
x = SG_Water_Catchbasin.columns
y = SG_Water_Catchbasin.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Catchbasin", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Discharge Chamber feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Discharge_Chamber = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDDSCH")
SG_Water_Discharge_Chamber.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | NDDSCH_KY | EASTING | NORTHING | DCH_OLD | COVER_LVL | INV_LVL | DEPTH | COV_SLB_LVL | RAISED | MH_TYPE | PS_AREA | PROJECT_REF | SURROUND | ... | CHAM_EXT_PROT_COND | CHAM_WALL_THKNS | CHAM_DIM_1 | CHAM_DIM_2 | CHAM_HEIGHT | SLB_SHAPE | SLB_COND | SLB_PROT | SLB_PROT_COND | SLB_DIM_1 | SLB_DIM_2 | SLB_O_DIM_1 | SLB_O_DIM_2 | SLB_THKNS | CONFIDENCE | RELIABLE | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 10401 | SGNDDSCH | IA 12/13 C 071 ST | As Built | None | None | None | 51.0 | COLLECTION/DISCHARGE CHAMBER | Excel | Manual | None | Drainage O&M | O&M Drainage | 2018-05-28 11:19:38 | ES | 2019-02-26 12:20:07 | None | None | None | 2018-05-28 11:20:32.000001 | ES/AH(FQC) | 2019-02-26 12:20:15 | None | None | Ashghal | None | NaN | 217349.881 | 399324.808 | TA33/2/CC | 29.520 | 26.047 | 3.473 | 28.910 | 610.0 | COLLECTION/DISCHARG* | None | C2014/127 | 7 | ... | 1 | 550.0 | 2800.0 | 3300.0 | 3.380 | 2 | 1 | 4 | 1 | 3900.0 | 4400.0 | 2800.0 | 3300.0 | 300.0 | None | None | 1 | {2348B553-C3B5-4D01-8BD8-639BB0DFFB9F} | 11222544 | None | 101 | None | 55002797 | OPERATING | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"x": 217349.88100000005, "y": 399324.80800000... |
| 1 | 10417 | SGNDDSCH | IA 10/11 C 117 G | As Built Yet to be Approved | None | Qatar North | None | 66.0 | None | Excel | Automation | None | Drainage O&M | O&M Drainage | 2021-07-06 00:00:00 | VV | NaT | None | None | None | NaT | VV/ES | 2021-07-06 00:00:00 | None | None | Ashghal | None | NaN | 230578.970 | 400525.179 | SW4C/08/DC | 3.285 | 0.760 | 2.525 | 3.185 | NaN | Discharge Chamber | None | C2012/18 | 2 | ... | 1 | 200.0 | 2000.0 | 1000.0 | 2.225 | 2 | 1 | None | None | 2400.0 | 1400.0 | 2000.0 | 1000.0 | 200.0 | 45 | 45 | 1 | {22702465-5F4F-4525-80BF-B45CE674A322} | 15965655 | None | 101 | None | 64080519 | OPERATING | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"x": 230578.96999999974, "y": 400525.17899999... |
2 rows × 111 columns
SG_Water_Discharge_Chamber.shape
(78, 111)
SG_Water_Discharge_Chamber_null = pd.DataFrame(SG_Water_Discharge_Chamber.isnull().sum())
SG_Water_Discharge_Chamber_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 4 |
| PROJECT_STATUS | 10 |
| RD_ANAME | 78 |
| ... | ... |
| CONTRACT_ID | 78 |
| PROJECT_ID | 78 |
| ASSET_TAG | 78 |
| ASSET_PRIORITY | 78 |
| SHAPE | 0 |
111 rows × 1 columns
x = SG_Water_Discharge_Chamber.columns
y = SG_Water_Discharge_Chamber.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Discharge Chamber", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Micro Tunnel Sewer Pipe feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Fittings = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDFITT")
SG_Water_Fittings.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | NDFITT_KY | EASTING | NORTHING | STATUS | SUBTYPE_CODE | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SGW_TYPE | FITTING_NUM | GROUND_LVL | INV_LVL | FITT_TYPE | FITT_MAT | FITT_SIZE1 | FITT_SIZE2 | REMARKS | PS_AREA | PROJ_REF | TAKAMUL_CATCHMENT | CONFIDENCE | RELIABLE | GLOBALID | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 17 | SGNDFITT | IA 12/13 C 058 ST | As Built | None | None | NaN | 54.0 | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-03-04 | IW | NaT | None | None | None | None | IW/EA | 2020-04-14 21:07:48 | None | None | Ashghal | None | None | 225028.1193 | 393536.7500 | CN | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | P | R6UP2/3/RTC-1/1 (45°BEND) | -10.29 | -10.787 | 5 | UP | 300 | 300.0 | None | None | C2013/89 | Qatar North | 45 | NaN | {D583808E-1748-46C7-A285-F74C18ACA18B} | {"x": 225028.11930000037, "y": 393536.75, "spa... |
| 1 | 19 | SGNDFITT | IA 12/13 C 058 ST | As Built | None | None | NaN | 54.0 | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-03-04 | IW | NaT | None | None | None | None | IW/EA | 2020-04-14 21:07:48 | None | None | Ashghal | None | None | 225070.1650 | 393553.6341 | CN | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | P | R6UP2/1/RTC-1/1 (45°BEND) | -10.21 | -10.720 | 5 | UP | 300 | 300.0 | None | None | C2013/89 | Qatar North | 45 | NaN | {07A5B0E4-B263-49F5-ABF3-1CD61D93564F} | {"x": 225070.16500000004, "y": 393553.63409999... |
SG_Water_Fittings.shape
(350, 68)
SG_Water_Fittings_null = pd.DataFrame(SG_Water_Fittings.isnull().sum())
SG_Water_Fittings_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 0 |
| PROJECT_STATUS | 0 |
| RD_ANAME | 334 |
| ... | ... |
| TAKAMUL_CATCHMENT | 0 |
| CONFIDENCE | 0 |
| RELIABLE | 170 |
| GLOBALID | 0 |
| SHAPE | 0 |
68 rows × 1 columns
x = SG_Water_Fittings.columns
y = SG_Water_Fittings.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Micro Tunnel Sewer Pipe", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Gully Connection End Cap feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Gully_Connec_End_Cap = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDGCST")
SG_Water_Gully_Connec_End_Cap.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | EASTING | NORTHING | GULLY_NO | COVER_LVL | INV_LVL | DEPTH | PROJECT_REF | STATUS | NDGULL_KY | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 21494 | SGNDGCST | CP05B | As Built | None | None | None | 69.0 | GW END CAP | Excel | Manual | None | Drainage O&M | O&M Drainage | 2018-09-16 14:27:43 | ES | 2018-09-16 14:27:58 | None | None | None | None | ES | 2018-09-16 14:28:13 | None | None | Ashghal | None | 228388.877 | 409326.028 | SA1/45/4/GW1 | 5.904 | 4.477 | 1.427 | 2888 | NC | None | 1 | {5F0D4627-18F5-4F36-B0EA-82E8E3590A17} | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"x": 228388.87700000033, "y": 409326.02800000... |
| 1 | 21491 | SGNDGCST | CP05B | As Built | None | None | None | 69.0 | GW END CAP | Excel | Manual | None | Drainage O&M | O&M Drainage | 2018-09-16 14:27:43 | ES | 2018-09-16 14:27:58 | None | None | None | None | ES | 2018-09-16 14:28:13 | None | None | Ashghal | None | 228429.244 | 409332.476 | SA1/45/5/GW2 | 5.610 | 4.194 | 1.416 | 2888 | NC | None | 1 | {7906F195-E46A-499B-A858-BEEE7134F5C0} | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"x": 228429.24399999995, "y": 409332.47599999... |
SG_Water_Gully_Connec_End_Cap.shape
(532, 60)
SG_Water_Gully_Connec_End_Cap_null = pd.DataFrame(SG_Water_Gully_Connec_End_Cap.isnull().sum())
SG_Water_Gully_Connec_End_Cap_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 0 |
| PROJECT_STATUS | 0 |
| RD_ANAME | 532 |
| RD_ENAME | 532 |
| STREET_NO | 532 |
| ZONE_NO | 2 |
| FEATURE_REMARKS | 30 |
| SOURCE_DATA_INPUT_FORMAT | 4 |
| DATA_INPUT_METHOD | 4 |
| METADATA | 532 |
| SOURCE_DEPT_SECTION_UNIT | 0 |
| OWNER | 0 |
| DATE_CREATED | 0 |
| EDITED_BY | 0 |
| LAST_UPDATED_DATE | 0 |
| BUSINESS_APPROVAL_BY | 532 |
| BUSINESS_APPROVAL_DATE | 532 |
| CONDITIONAL_DATA_APPROVAL_BY | 532 |
| CONDITIONAL_DATA_APPROVAL_DATE | 532 |
| DATA_QC_APPROVAL_BY | 4 |
| DATA_QC_APPROVAL_DATE | 4 |
| COST_PER_ITEM_METER | 532 |
| STOCK_ON_HAND | 532 |
| AGENCY_NAME | 0 |
| DMS_CODE | 532 |
| EASTING | 0 |
| NORTHING | 0 |
| GULLY_NO | 36 |
| COVER_LVL | 0 |
| INV_LVL | 0 |
| DEPTH | 0 |
| PROJECT_REF | 0 |
| STATUS | 0 |
| NDGULL_KY | 532 |
| SUBTYPE_CODE | 0 |
| GLOBALID | 0 |
| MXASSETNUM | 532 |
| MXLOCATION | 532 |
| MXSITEID | 532 |
| MXCREATIONSTATE | 532 |
| ROWSTAMP | 0 |
| MXSTATUS | 532 |
| PARENT_GLOBALID | 532 |
| LOCATION_GLOBAL_ID | 532 |
| DATA_LOAD_ID | 532 |
| DISCIPLINE | 532 |
| GPS_X | 532 |
| GPS_Y | 532 |
| GPS_Z | 532 |
| OPERATOR | 532 |
| MAINTAINER | 532 |
| MUNICIPALITY | 532 |
| DISTRICT | 532 |
| CONTRACT_ID | 532 |
| PROJECT_ID | 532 |
| ASSET_TAG | 532 |
| ASSET_PRIORITY | 532 |
| SHAPE | 0 |
x = SG_Water_Gully_Connec_End_Cap.columns
y = SG_Water_Gully_Connec_End_Cap.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Gully Connection End Cap", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Gully feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Gully = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDGULL")
SG_Water_Gully.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | EASTING | NORTHING | GULLY_NO | COVER_LVL | INV_LVL | DEPTH | PROJECT_REF | STATUS | NDGULL_KY | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 4214793 | SGNDGULL | ERC1400/D20 | As Built | 0.0 | 0.0 | None | DWG | Data Conversion | 2013-12-05 | Lateef | NaT | None | None | O & M | NaT | EJAZ | NaT | None | None | Ashghal | None | NaN | NaN | SW/9/A1/G3 | 0.0 | 0.0 | NaN | ERC1400/D20 | Not Approved | NaN | 1 | {5F09E89C-EC58-4D1B-9F12-763D3D33F5F7} | 624123 | None | 101 | None | 63316473 | OPERATING | None | None | None | None | NaN | NaN | NaN | None | None | None | None | None | None | None | 1.0 | {"x": 229223.83640000038, "y": 389127.69209999... | |||||
| 1 | 4212780 | SGNDGULL | ERC1400/D20 | As Built | 0.0 | 0.0 | None | DWG | Data Conversion | 2013-12-05 | Lateef | NaT | None | None | O & M | NaT | EJAZ | NaT | None | None | Ashghal | None | NaN | NaN | SW/9/6/9/G4 | 0.0 | 0.0 | NaN | ERC1400/D20 | Not Approved | NaN | 1 | {81BEB78C-E0C7-4E1E-93E9-57BB0618E03D} | 621618 | None | 101 | None | 63315957 | OPERATING | None | None | None | None | NaN | NaN | NaN | None | None | None | None | None | None | None | 1.0 | {"x": 229369.06510000024, "y": 389137.56890000... |
SG_Water_Gully.shape
(172657, 60)
SG_Water_Gully_null = pd.DataFrame(SG_Water_Gully.isnull().sum())
SG_Water_Gully_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 65 |
| PROJECT_CODE | 15572 |
| PROJECT_STATUS | 15619 |
| RD_ANAME | 163614 |
| RD_ENAME | 124492 |
| STREET_NO | 155240 |
| ZONE_NO | 34177 |
| FEATURE_REMARKS | 88473 |
| SOURCE_DATA_INPUT_FORMAT | 49 |
| DATA_INPUT_METHOD | 45 |
| METADATA | 166561 |
| SOURCE_DEPT_SECTION_UNIT | 2810 |
| OWNER | 48 |
| DATE_CREATED | 2830 |
| EDITED_BY | 1 |
| LAST_UPDATED_DATE | 86538 |
| BUSINESS_APPROVAL_BY | 172655 |
| BUSINESS_APPROVAL_DATE | 172655 |
| CONDITIONAL_DATA_APPROVAL_BY | 136664 |
| CONDITIONAL_DATA_APPROVAL_DATE | 134886 |
| DATA_QC_APPROVAL_BY | 870 |
| DATA_QC_APPROVAL_DATE | 40016 |
| COST_PER_ITEM_METER | 172657 |
| STOCK_ON_HAND | 172657 |
| AGENCY_NAME | 397 |
| DMS_CODE | 169497 |
| EASTING | 826 |
| NORTHING | 826 |
| GULLY_NO | 2112 |
| COVER_LVL | 4500 |
| INV_LVL | 5594 |
| DEPTH | 6127 |
| PROJECT_REF | 15928 |
| STATUS | 2385 |
| NDGULL_KY | 150673 |
| SUBTYPE_CODE | 0 |
| GLOBALID | 0 |
| MXASSETNUM | 0 |
| MXLOCATION | 172640 |
| MXSITEID | 0 |
| MXCREATIONSTATE | 172657 |
| ROWSTAMP | 0 |
| MXSTATUS | 0 |
| PARENT_GLOBALID | 172657 |
| LOCATION_GLOBAL_ID | 172657 |
| DATA_LOAD_ID | 172657 |
| DISCIPLINE | 172640 |
| GPS_X | 172640 |
| GPS_Y | 172640 |
| GPS_Z | 172640 |
| OPERATOR | 172640 |
| MAINTAINER | 172640 |
| MUNICIPALITY | 172572 |
| DISTRICT | 172555 |
| CONTRACT_ID | 169361 |
| PROJECT_ID | 172640 |
| ASSET_TAG | 172640 |
| ASSET_PRIORITY | 25774 |
| SHAPE | 0 |
x = SG_Water_Gully.columns
y = SG_Water_Gully.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Gully", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Micro Tunnel Sewer Pipe feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Gully = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDGULL")
SG_Water_Gully.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | EASTING | NORTHING | GULLY_NO | COVER_LVL | INV_LVL | DEPTH | PROJECT_REF | STATUS | NDGULL_KY | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 4212580 | SGNDGULL | IA 13/14 C 036 G | As Built | None | None | NaN | 27.0 | As per SR#10167204 Existing Gully is abandoned | Excel | Manual | None | Drainage O&M | O&M Drainage | 2018-09-23 | ES/BD | 2021-06-22 00:00:00 | None | NaT | None | NaT | NaT | None | None | Ashghal | None | 233891.0970 | 391199.2280 | MB29/CP/10/G2 | 10.47 | 9.52 | 0.95 | C2014/02 | AB | NaN | 2 | {202DDCE2-452D-4A25-9793-BD99966AAD1C} | 12328306 | None | 101 | None | 63339748 | DECOMM | None | None | None | None | None | None | None | None | None | None | None | None | None | None | 1.0 | {"x": 233891.09700000007, "y": 391199.22800000... | |
| 1 | 4212354 | SGNDGULL | None | None | None | None | NaN | NaN | Unknown asset captured for beneficiary purpose... | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-10-03 | VV | 2020-10-04 07:26:22 | None | NaT | None | NaT | VV/ES | 2020-10-03 | None | None | Ashghal | None | 224160.3275 | 398842.7383 | G | NaN | NaN | NaN | None | Disposed | NaN | 3 | {BC120F7D-9F53-4908-ABA8-F95ACBB4C4C1} | 13936079 | None | 101 | None | 62743730 | DISPOSED | None | None | None | None | None | None | None | None | None | None | None | None | None | None | NaN | {"x": 224160.32749999966, "y": 398842.73829999... |
SG_Water_Gully.shape
(172657, 60)
SG_Water_Gully_null = pd.DataFrame(SG_Water_Gully.isnull().sum())
SG_Water_Gully_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 65 |
| PROJECT_CODE | 15572 |
| PROJECT_STATUS | 15619 |
| RD_ANAME | 163614 |
| RD_ENAME | 124492 |
| STREET_NO | 155240 |
| ZONE_NO | 34177 |
| FEATURE_REMARKS | 88473 |
| SOURCE_DATA_INPUT_FORMAT | 49 |
| DATA_INPUT_METHOD | 45 |
| METADATA | 166561 |
| SOURCE_DEPT_SECTION_UNIT | 2810 |
| OWNER | 48 |
| DATE_CREATED | 2830 |
| EDITED_BY | 1 |
| LAST_UPDATED_DATE | 86538 |
| BUSINESS_APPROVAL_BY | 172655 |
| BUSINESS_APPROVAL_DATE | 172655 |
| CONDITIONAL_DATA_APPROVAL_BY | 136664 |
| CONDITIONAL_DATA_APPROVAL_DATE | 134886 |
| DATA_QC_APPROVAL_BY | 870 |
| DATA_QC_APPROVAL_DATE | 40016 |
| COST_PER_ITEM_METER | 172657 |
| STOCK_ON_HAND | 172657 |
| AGENCY_NAME | 397 |
| DMS_CODE | 169497 |
| EASTING | 826 |
| NORTHING | 826 |
| GULLY_NO | 2112 |
| COVER_LVL | 4500 |
| INV_LVL | 5594 |
| DEPTH | 6127 |
| PROJECT_REF | 15928 |
| STATUS | 2385 |
| NDGULL_KY | 150673 |
| SUBTYPE_CODE | 0 |
| GLOBALID | 0 |
| MXASSETNUM | 0 |
| MXLOCATION | 172640 |
| MXSITEID | 0 |
| MXCREATIONSTATE | 172657 |
| ROWSTAMP | 0 |
| MXSTATUS | 0 |
| PARENT_GLOBALID | 172657 |
| LOCATION_GLOBAL_ID | 172657 |
| DATA_LOAD_ID | 172657 |
| DISCIPLINE | 172640 |
| GPS_X | 172640 |
| GPS_Y | 172640 |
| GPS_Z | 172640 |
| OPERATOR | 172640 |
| MAINTAINER | 172640 |
| MUNICIPALITY | 172572 |
| DISTRICT | 172555 |
| CONTRACT_ID | 169361 |
| PROJECT_ID | 172640 |
| ASSET_TAG | 172640 |
| ASSET_PRIORITY | 25774 |
| SHAPE | 0 |
x = SG_Water_Gully.columns
y = SG_Water_Gully.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Micro Tunnel Sewer Pipe", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Internal Backdrop feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Internal_Backdrop = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDINBD")
SG_Water_Internal_Backdrop.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | NDMNHL_KY | EASTING | NORTHING | STATUS | SUBTYPE_CODE | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SGW_TYPE | SH_MH_NO | INT_BD_NO | CAP_LVL | IN_PIPE_IL | OUT_PIPE_IL | SOURCE_MH | INT_PIPE_DIA | INT_PIPE_MAT | PIPE_LEN | PS_AREA | PROJ_REF | REMARKS | TAKAMUL_CATCHMENT | CONFIDENCE | RELIABLE | GLOBALID | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 15 | SGNDINBD | IA 11/12 C 02 G | As Built | None | None | None | 56 | Excel | Manual | None | Drainage O&M | O&M Drainage | 2020-04-13 13:32:19 | ES | 2020-04-13 13:32:20 | None | None | None | None | ES | 2020-04-13 13:32:21.000001 | None | None | Ashghal | None | None | 230919.074 | 385223.523 | CN | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | P | SW10/17 | SW10/17/IBD1 | 12.14 | -1.706 | -11.456 | SW10/17/A1 | 1000 | GP | 11.463 | None | C2012/74 | None | Qatar South | 100 | None | {D58587C4-40F3-43F1-9978-00E4791EE649} | {"x": 230919.07400000002, "y": 385223.52300000... |
| 1 | 1 | SGNDINBD | IA 11/12 C 02 G | As Built | None | None | None | 49 | Excel | Manual | None | Drainage O&M | O&M Drainage | 2020-04-13 13:32:19 | ES | 2020-04-13 13:32:20 | None | None | None | None | ES | 2020-04-13 13:32:21.000001 | None | None | Ashghal | None | None | 238193.949 | 387379.959 | CN | 1 | None | None | None | None | 0 | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | P | SW10/2\n | SW10/2/IBD1 | 3.57 | 0.311 | -13.764 | SW10/2/1 | 1000 | GP | 15.777 | None | C2012/74 | None | Qatar West | 100 | None | {23475FEA-2AD2-405E-98AB-72609129D7E4} | {"x": 238193.62090000045, "y": 387379.98020000... |
SG_Water_Internal_Backdrop.shape
(15, 70)
SG_Water_Internal_Backdrop_null = pd.DataFrame(SG_Water_Internal_Backdrop.isnull().sum())
SG_Water_Internal_Backdrop_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 0 |
| PROJECT_STATUS | 0 |
| RD_ANAME | 15 |
| ... | ... |
| TAKAMUL_CATCHMENT | 0 |
| CONFIDENCE | 0 |
| RELIABLE | 15 |
| GLOBALID | 0 |
| SHAPE | 0 |
70 rows × 1 columns
x = SG_Water_Internal_Backdrop.columns
y = SG_Water_Internal_Backdrop.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Internal Backdrop", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
SGW Deep Injection well feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SGW_Deep_Injection_well = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDINJW")
SGW_Deep_Injection_well.head(2)
| OBJECTID | GFCODE | PROJECT_REF | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | NETWORK_TYPE | PROJECT_SITE | WELL_NAME | WELL_TYPE | X_COORD | Y_COORD | DEPTH | DIAMETER | CAPACITY | REMARKS | STATUS | NDINJW_KY | REF_ID | CONST_DATE | CONFIDENCE | RELIABLE | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 49723 | SGNDINJW | 0.0 | 0.0 | 1899-12-30 | 1899-12-30 | 1899-12-30 | 1899-12-30 | 1899-12-30 | 0.0 | 0.0 | Doh_MW50_077 | Monitoring | 234014.999659 | 388046.000352 | 50 | 0.0 | 0.0 | Existing data Updated as per SR-10169771 | Operational | 0.0 | 150077 | 1899-12-30 | 0.0 | 0.0 | 1 | {37168AFE-B3F2-4A37-831A-8DFCCE2FB11D} | 628207 | None | 101 | None | 62193393 | OPERATING | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"x": 234014.99969999958, "y": 388046.00039999... | |||||||||||||||||||
| 1 | 49720 | SGNDINJW | 0.0 | 0.0 | 1899-12-30 | 1899-12-30 | 1899-12-30 | 1899-12-30 | 1899-12-30 | 0.0 | 0.0 | Doh_MW50_085 | Monitoring | 195848.999657 | 389252.000346 | 50 | 0.0 | 0.0 | Existing data Updated as per SR-10169771 | Operational | 0.0 | 150085 | 1899-12-30 | 0.0 | 0.0 | 1 | {B6EA6250-9192-4FC9-A563-8829F3939D01} | 628211 | None | 101 | None | 62193390 | OPERATING | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"x": 195848.99969999958, "y": 389252.00029999... |
SG_Water_Internal_Backdrop.shape
(15, 70)
SGW_Deep_Injection_well_null = pd.DataFrame(SG_Water_Internal_Backdrop.isnull().sum())
SGW_Deep_Injection_well_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 0 |
| PROJECT_STATUS | 0 |
| RD_ANAME | 15 |
| ... | ... |
| TAKAMUL_CATCHMENT | 0 |
| CONFIDENCE | 0 |
| RELIABLE | 15 |
| GLOBALID | 0 |
| SHAPE | 0 |
70 rows × 1 columns
x = SG_Water_Pond.columns
y = SG_Water_Pond.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("SGW Deep Injection well", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Manhole feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SGW_Water_Manhole = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDMNHL")
SGW_Water_Manhole.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | NDMNHL_KY | EASTING | NORTHING | SGW_OLD | COVER_LVL | INV_LVL | DEPTH | COV_SLB_LVL | RAISED | PS_AREA | PROJECT_REF | MH_TYPE | SURROUND | ... | RS_DIM_1 | RS_DIM_2 | RS_O_DIM_1 | RS_O_DIM_2 | RS_SLB_THKNS | RP_SHAPE | RP_COND | RP_SLB_PROT | RP_PROT_COND | RP_DIM_1 | RP_DIM_2 | RP_O_DIM_1 | RP_O_DIM_2 | RP_SLB_THKNS | CONFIDENCE | RELIABLE | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 2173681 | SGNDSOKW | <Null> | As Built Yet to be Approved | None | None | NaN | 66.0 | Captured from the project C2016/50 SR022 | Excel | Automation | None | Drainage O&M | O&M Drainage | 2019-11-24 09:26:42 | VV | NaT | None | None | None | NaT | VV/EA | 2019-11-25 11:35:36 | None | None | Ashghal | None | NaN | 230417.559 | 400525.044 | 66/517/SA2 | 2.830 | NaN | NaN | NaN | NaN | None | <Null> | None | None | ... | NaN | NaN | NaN | NaN | NaN | None | None | None | None | NaN | NaN | NaN | NaN | NaN | None | None | 1 | {9BD2AC65-B64D-460F-9C87-5FA49AD80BA2} | 12935842 | None | 101 | None | 54934243 | OPERATING | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"x": 230417.55900000036, "y": 400525.04399999... |
| 1 | 2174493 | SGNDMNHL | IA 2016 C 013 G | As Built | None | None | NaN | 68.0 | None | Excel | Automation | None | Drainage O&M | O&M Drainage | 2020-07-09 00:00:00 | IW | 2020-07-12 11:28:54 | None | None | None | NaT | ES | 2020-07-12 11:28:58 | None | None | Ashghal | None | NaN | 228354.978 | 402398.842 | 68/233/TR5/3 | 4.199 | 0.72 | 3.479 | 3.749 | 450.0 | None | C2016/54 | None | 1 | ... | NaN | NaN | NaN | NaN | NaN | None | None | None | None | NaN | NaN | NaN | NaN | NaN | 80 | None | 1 | {4CFAD8A5-4804-4B79-8712-DC46B702977D} | 13487273 | None | 101 | None | 54935484 | OPERATING | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"x": 228354.97800000012, "y": 402398.84200000... |
2 rows × 142 columns
SGW_Water_Manhole.shape
(78460, 142)
SGW_Water_Manhole_null = pd.DataFrame(SGW_Water_Manhole.isnull().sum())
SGW_Water_Manhole_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 61 |
| PROJECT_CODE | 8902 |
| PROJECT_STATUS | 5526 |
| RD_ANAME | 75522 |
| ... | ... |
| CONTRACT_ID | 77731 |
| PROJECT_ID | 78458 |
| ASSET_TAG | 78458 |
| ASSET_PRIORITY | 78460 |
| SHAPE | 0 |
142 rows × 1 columns
x = SGW_Water_Manhole.columns
y = SGW_Water_Manhole.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Manhole", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Micro Tunnel Manhole feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SGW_Water_MT_Manhole = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDMTMH")
SGW_Water_MT_Manhole.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | ARMTMH_KY | SH_MH_NO | PS_AREA | PROJECT_REF | SURROUND | SURFACE | CONST_DATE | COV_A_EAST | COV_A_NORT | COV_A_LVL | COV_B_EAST | COV_B_NORT | COV_B_LVL | ... | LNDG_SLB_C_SFT_TYPE | WATER_TABLE | TURBULENT | INFILTRATION | STATUS | BENCH_FORM | BENCH_COND | BENCH_PROT | CORNER_A_EAST | CORNER_A_NORT | CORNER_B_EAST | CORNER_B_NORT | CORNER_C_EAST | CORNER_C_NORT | CORNER_D_EAST | CORNER_D_NORT | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 37238 | SGNDMTMH | CP07A-1A | As Built | None | None | NaN | 69.0 | None | Excel | Manual | None | Drainage O&M | O&M Drainage | 2019-02-03 08:43:11 | ES | 2019-02-03 08:43:16 | None | None | None | NaT | ES | 2019-02-03 08:43:23 | None | None | Ashghal | None | None | GL1/2/12 | GL1 | 4523 | 1 | 1 | 2016-02-15 | 229874.497 | 406214.882 | 3.218 | NaN | NaN | NaN | ... | None | B | N | N | Connected / Live | Y | 1 | 3 | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 1 | {278A6947-8B7F-4266-82AB-B775C995AEB3} | 12384459 | None | 101 | None | 54754990 | OPERATING | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"x": 229874.49700000044, "y": 406214.88199999... |
| 1 | 37236 | SGNDMTMH | CP07A-1A | As Built | None | None | NaN | 69.0 | None | Excel | Manual | None | Drainage O&M | O&M Drainage | 2019-02-03 08:43:11 | ES | 2019-02-03 08:43:16 | None | None | None | NaT | ES | 2019-02-03 08:43:23 | None | None | Ashghal | None | None | GL1/2/10 | GL1 | 4523 | 1 | 1 | 2016-02-15 | 229874.298 | 406187.195 | 3.316 | NaN | NaN | NaN | ... | None | B | N | N | Connected / Live | Y | 1 | 3 | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | 1 | {D3EB1A2B-3473-44A4-AD33-C2CF1CD0C8D3} | 12384928 | None | 101 | None | 54754964 | OPERATING | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"x": 229874.29800000042, "y": 406187.19500000... |
2 rows × 166 columns
SGW_Water_MT_Manhole.shape
(1200, 166)
SGW_Water_MT_Manhole_null = pd.DataFrame(SGW_Water_MT_Manhole.isnull().sum())
SGW_Water_MT_Manhole_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 12 |
| PROJECT_STATUS | 2 |
| RD_ANAME | 1098 |
| ... | ... |
| CONTRACT_ID | 1178 |
| PROJECT_ID | 1200 |
| ASSET_TAG | 1200 |
| ASSET_PRIORITY | 1200 |
| SHAPE | 0 |
166 rows × 1 columns
x = SGW_Water_MT_Manhole.columns
y = SGW_Water_MT_Manhole.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Micro Tunnel Manhole", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Outfall Chamber feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SGW_Water_Outfall_Chamber = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDOTFL")
SGW_Water_Outfall_Chamber.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | NDOTFL_KY | OUTFALL_NUM | PS_NUM | COVER_LVL | PROJECT_REF | EASTING | NORTHING | SUBTYPE_CODE | GLOBALID | STATUS | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 11399 | SGNDOTFL | AA/DOM/13-14/M/1591-4.07 | As Built Yet to be Approved | None | None | NaN | NaN | None | A | Automation | None | None | O&M Drainage | 2019-12-07 | VB | NaT | None | None | None | NaT | VB/EA | 2019-12-24 18:31:51 | None | None | Ashghal | None | NaN | 51/1755/EFA23/OF1 | None | NaN | C2018/88 | 214956.070 | 398550.593 | 1 | {AA165FF3-386E-4414-8204-95AB23CA6971} | CN | 13037480 | None | 101 | None | 55002716 | OPERATING | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"x": 214956.0700000003, "y": 398550.593000000... |
| 1 | 11316 | SGNDOTFL | AA DOM13-14 M1591-4.07/G | As Built | None | None | NaN | NaN | None | Excel | Automation | None | Drainage O&M | O&M Drainage | 2019-07-11 | VV | NaT | None | None | None | NaT | BD | 2019-07-15 00:00:00 | None | None | Ashghal | None | NaN | GS/EFA1/OF2 | None | NaN | C2017/36 | 222400.585 | 398769.109 | 1 | {76600E8B-D854-4B8E-A3BC-0446AD1CA44C} | CN | 12728120 | None | 101 | None | 55002629 | OPERATING | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"x": 222400.58499999996, "y": 398769.10899999... |
SGW_Water_Outfall_Chamber.shape
(190, 59)
SGW_Water_Outfall_Chamber_null = pd.DataFrame(SGW_Water_Outfall_Chamber.isnull().sum())
SGW_Water_Outfall_Chamber_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 14 |
| PROJECT_STATUS | 14 |
| RD_ANAME | 189 |
| RD_ENAME | 188 |
| STREET_NO | 189 |
| ZONE_NO | 128 |
| FEATURE_REMARKS | 165 |
| SOURCE_DATA_INPUT_FORMAT | 13 |
| DATA_INPUT_METHOD | 13 |
| METADATA | 189 |
| SOURCE_DEPT_SECTION_UNIT | 34 |
| OWNER | 13 |
| DATE_CREATED | 13 |
| EDITED_BY | 13 |
| LAST_UPDATED_DATE | 134 |
| BUSINESS_APPROVAL_BY | 190 |
| BUSINESS_APPROVAL_DATE | 190 |
| CONDITIONAL_DATA_APPROVAL_BY | 188 |
| CONDITIONAL_DATA_APPROVAL_DATE | 188 |
| DATA_QC_APPROVAL_BY | 13 |
| DATA_QC_APPROVAL_DATE | 13 |
| COST_PER_ITEM_METER | 190 |
| STOCK_ON_HAND | 190 |
| AGENCY_NAME | 0 |
| DMS_CODE | 190 |
| NDOTFL_KY | 180 |
| OUTFALL_NUM | 2 |
| PS_NUM | 169 |
| COVER_LVL | 58 |
| PROJECT_REF | 14 |
| EASTING | 13 |
| NORTHING | 13 |
| SUBTYPE_CODE | 0 |
| GLOBALID | 0 |
| STATUS | 0 |
| MXASSETNUM | 0 |
| MXLOCATION | 190 |
| MXSITEID | 0 |
| MXCREATIONSTATE | 190 |
| ROWSTAMP | 0 |
| MXSTATUS | 0 |
| PARENT_GLOBALID | 190 |
| LOCATION_GLOBAL_ID | 190 |
| DATA_LOAD_ID | 190 |
| DISCIPLINE | 190 |
| GPS_X | 190 |
| GPS_Y | 190 |
| GPS_Z | 190 |
| OPERATOR | 190 |
| MAINTAINER | 190 |
| MUNICIPALITY | 190 |
| DISTRICT | 190 |
| CONTRACT_ID | 190 |
| PROJECT_ID | 190 |
| ASSET_TAG | 190 |
| ASSET_PRIORITY | 190 |
| SHAPE | 0 |
x = SGW_Water_Outfall_Chamber.columns
y = SGW_Water_Outfall_Chamber.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Outfall Chamber", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Pumping Station feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Pumping_Station = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDPSTN")
SG_Water_Pumping_Station.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ENAME | RD_ANAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | PROJECT_REF | PS_NUM | NDPSTN_KY | TEL | ELEC_SER_NUM | TELEMETRY_PHONE | WATER_SER_NUM | PLOT_NUM | PS_TYPE | STATUS | WATER_TABLE | PLANNED_PUMPS | STATIC_HEAD | OPERTNL_PUMPS | GROUND_LVL | PUMP_ROOM_FLR_LVL | WET_WELL_FLR_LVL | MOTOR_ROOM_FLR | FIRE_DETECTION | FIRE_CONTROL | OVERPUMP_CONN | SIZE_OF_CONN | FLOW_METER | RISING_MAINS | WET_WELLS | CONFIDENCE | RELIABLE | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 11022 | SGNDPSTN | None | As Built Yet to be Approved | None | None | NaN | 12.0 | Al Rumailah Underpass Pumping station [This PS... | Excel | Automation | None | Drainage O&M | ASHGHAL | 2019-12-30 | ES/BH | NaT | None | None | None | NaT | ES | 2019-12-30 13:59:16 | None | None | Ashghal | None | None | RM2 | NaN | NaN | None | NaN | NaN | NaN | None | CN | None | NaN | NaN | NaN | NaN | NaN | NaN | NaN | None | None | None | NaN | None | NaN | NaN | 75.0 | NaN | 1 | {E08D704E-CB4A-43A0-AC4F-AAC505B487EB} | None | 109125 | 101 | None | 55001977 | OPERATING | None | None | None | None | None | None | None | 101-201051-DOHA | 101-201051-DOHA | None | None | C/2020/47 | None | DA12-PSG-RM2 | NaN | {"x": 230254.1202999996, "y": 393698.369500000... |
| 1 | 11050 | SGNDPSTN | DSC0014 | None | Huwar | None | 281.0 | 52.0 | Existing PS is abandoned according to the proj... | None | None | None | Drainage O&M | ASHGHAL | NaT | AH/PC/BD | 2021-01-18 | None | None | None | NaT | None | NaT | None | None | None | None | C366 | LAQTA PS | 16.0 | NaN | None | NaN | NaN | NaN | UC | None | NaN | NaN | NaN | NaN | NaN | NaN | NaN | None | None | None | NaN | None | NaN | NaN | 0.0 | 0.0 | 1 | {67E34D2F-123B-4576-91AD-AEC6509739F6} | None | 106747 | 101 | None | 61868279 | NOT READY | None | None | None | None | None | None | None | None | None | None | None | None | None | RA52-PSG-LAQTA | NaN | {"x": 224176.99260000046, "y": 394818.48420000... |
SG_Water_Pumping_Station.shape
(98, 78)
SG_Water_Pumping_Station_null = pd.DataFrame(SG_Water_Pumping_Station.isnull().sum())
SG_Water_Pumping_Station_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 43 |
| PROJECT_STATUS | 36 |
| RD_ENAME | 74 |
| ... | ... |
| CONTRACT_ID | 27 |
| PROJECT_ID | 98 |
| ASSET_TAG | 5 |
| ASSET_PRIORITY | 76 |
| SHAPE | 0 |
78 rows × 1 columns
x = SG_Water_Pumping_Station.columns
y = SG_Water_Pumping_Station.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Pumping Station", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Stub End (End Cap) feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Stub_End_Cap = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDSTUB")
SG_Water_Stub_End_Cap.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | NDSTUB_KY | PS_AREA | PROJECT_REF | EASTING | NORTHING | SGW_OLD | COVER_LVL | INV_LVL | DEPTH | CONST_DATE | CONFIDENCE | RELIABLE | SUBTYPE_CODE | GLOBALID | STATUS | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 28018 | SGNDSTUB | IA 14/15 C 0820 ST | As Built | None | None | NaN | 91.0 | STORMTECH ENDCAP DIMENSION WIDTH=1295mm, HEIGH... | Excel | Manual | None | Drainage O&M | O&M Drainage | 2018-03-28 | ES | 2018-03-28 | None | None | None | NaT | ES/AH(FQC) | 2018-03-28 | None | None | Ashghal | None | None | None | 219138.2549 | 374820.7474 | 91/NIA/SA/WAS/60/ST/1 | 29.921 | 29.177 | 0.744 | 2015-10-31 | 100.0 | 100.0 | 1 | {82DD8BA0-EACD-4400-A026-613D33BDED74} | CN | None | None | None | None | None | None | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"x": 219138.25490000006, "y": 374820.74740000... | |
| 1 | 28016 | SGNDSTUB | IA 14/15 C 0820 ST | As Built | None | None | NaN | 91.0 | STORMTECH ENDCAP DIMENSION WIDTH=1295mm, HEIGH... | Excel | Manual | None | Drainage O&M | O&M Drainage | 2018-03-28 | ES | 2018-03-28 | None | None | None | NaT | ES/AH(FQC) | 2018-03-28 | None | None | Ashghal | None | None | None | 218854.6974 | 374629.1946 | 91/NIA/SA/WAS/62/ST | 30.262 | 29.518 | 0.744 | 2015-10-20 | 100.0 | 100.0 | 1 | {4EDE20C5-4F27-43E5-988D-4A62C55C3E8B} | CN | None | None | None | None | None | None | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"x": 218854.69739999995, "y": 374629.19459999... |
SG_Water_Stub_End_Cap.shape
(856, 64)
SG_Water_Stub_End_Cap_null = pd.DataFrame(SG_Water_Stub_End_Cap.isnull().sum())
SG_Water_Stub_End_Cap_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 1 |
| PROJECT_CODE | 11 |
| PROJECT_STATUS | 1 |
| RD_ANAME | 806 |
| ... | ... |
| CONTRACT_ID | 855 |
| PROJECT_ID | 856 |
| ASSET_TAG | 856 |
| ASSET_PRIORITY | 856 |
| SHAPE | 0 |
64 rows × 1 columns
x = SG_Water_Stub_End_Cap.columns
y = SG_Water_Stub_End_Cap.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Stub End (End Cap)", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Holding Tanks feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Holding_Tanks = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDTANK")
SG_Water_Holding_Tanks.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | TANK_NUM | TANK_TYPE | PROJECT_REF | CONST_DATE | COV_A_EAST | COV_A_NORT | COV_B_EAST | COV_B_NORT | COV_C_EAST | COV_C_NORT | COV_D_EAST | COV_D_NORT | COV_A_LVL | ... | OT_IL_US_MH_C_INV_LVL | OT_IL_PIPE_C_BD_LVL | OT_IL_PIPE_C_MATERIAL | DESCRIPTION_C | OT_IL_C_EASTING | OT_IL_C_NORTHING | OL_MH_NUM | OL_PIPE_DIA | OL_DEPTH | OL_BDROP | OL_INV_LVL | OL_PIPE_MATERIAL | DESCRIPTION_OL | OL_EASTING | OL_NORTHING | SUBTYPE_CODE | GLOBALID | STATUS | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 11722 | SGNDATTK | IA 13/14 C 006 ST | As Built | None | None | NaN | 70.0 | None | Excel | Manual | None | Drainage O&M | O&M Drainage | 2017-12-21 11:10:08.000001 | ELANGO | 2017-12-21 11:10:22 | None | None | ELANGO | 2017-12-21 11:10:36 | ELANGO | 2017-12-21 11:10:50 | None | None | Ashghal | None | DN/ATTK05 | 1 | C2013/73 | 1905-07-07 | 224770.794 | 408513.4710 | NaN | NaN | NaN | NaN | NaN | NaN | 11.339 | ... | NaN | None | None | None | NaN | NaN | None | NaN | NaN | None | NaN | None | None | NaN | NaN | 1 | {2EA93632-33E2-4B5A-AA7B-26EC2FE53505} | CN | None | None | None | None | 0 | None | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"x": 224770.34269999992, "y": 408507.67119999... |
| 1 | 11719 | SGNDATTK | IA 12/13 C 071 ST | As Built | None | None | NaN | 51.0 | None | Excel | Manual | None | Drainage O&M | O&M Drainage | 2018-05-28 12:18:11.000000 | ES | 2019-02-27 10:48:01 | None | None | None | NaT | ES/AH(FQC) | 2019-02-27 10:48:11 | None | None | Ashghal | None | TA 33/1 | 1 | C2014/127 | 2017-07-29 | 217695.444 | 399160.1965 | 217688.9037 | 399156.4578 | 217681.82 | 399151.154 | 217684.4038 | 399134.351 | 31.490 | ... | NaN | None | None | None | NaN | NaN | TANK TA33/1 MH-1 | 300.0 | 3.08 | None | 27.85 | Vitrified Clay | None | 217692.181 | 399164.53 | 1 | {78CCC812-B76B-43AB-8702-3C0B116DA9A5} | Operational | None | None | None | None | 0 | None | None | None | None | None | NaN | NaN | None | None | None | None | None | None | None | None | None | {"x": 217699.9791000001, "y": 399134.612299999... |
2 rows × 220 columns
SG_Water_Holding_Tanks.shape
(314, 220)
SG_Water_Holding_Tanks_null = pd.DataFrame(SG_Water_Holding_Tanks.isnull().sum())
SG_Water_Holding_Tanks_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 5 |
| PROJECT_STATUS | 31 |
| RD_ANAME | 312 |
| ... | ... |
| CONTRACT_ID | 314 |
| PROJECT_ID | 314 |
| ASSET_TAG | 314 |
| ASSET_PRIORITY | 314 |
| SHAPE | 0 |
220 rows × 1 columns
x = SG_Water_Holding_Tanks.columns
y = SG_Water_Holding_Tanks.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Holding Tanks", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()
Surface Ground Water Valve feature class has:
• Attribute Fields (Columns): 41
• Total Records (Rows): 704,473
SG_Water_Valve = pd.DataFrame.spatial.from_featureclass(r"C:\Users\esd_kgathogo\AppData\Roaming\ESRI\Desktop10.7\ArcCatalog\PWAROADS.sde\SDE.DRG_SURFACE_GROUND_WATER_NETWORK\SDE.SGNDVALV")
SG_Water_Valve.head(2)
| OBJECTID | GFCODE | PROJECT_CODE | PROJECT_STATUS | RD_ANAME | RD_ENAME | STREET_NO | ZONE_NO | FEATURE_REMARKS | SOURCE_DATA_INPUT_FORMAT | DATA_INPUT_METHOD | METADATA | SOURCE_DEPT_SECTION_UNIT | OWNER | DATE_CREATED | EDITED_BY | LAST_UPDATED_DATE | BUSINESS_APPROVAL_BY | BUSINESS_APPROVAL_DATE | CONDITIONAL_DATA_APPROVAL_BY | CONDITIONAL_DATA_APPROVAL_DATE | DATA_QC_APPROVAL_BY | DATA_QC_APPROVAL_DATE | COST_PER_ITEM_METER | STOCK_ON_HAND | AGENCY_NAME | DMS_CODE | NDVALV_KY | VC_OLD | PROJECT_REF | STATUS | VALVE | BOLT_NO | VALVE_MANUFACTURER | VALVE_TYPE | VALVE_SIZE | CEN_LINE_DEPTH | COLOUR_CODE | VALVE_COND | SURFACE_BOX_COND | BYPASS | COATING | COAT_COND | VALVE_MATERIAL | VALVE_JOINT | EASTING | NORTHING | SUBTYPE_CODE | GLOBALID | MXASSETNUM | MXLOCATION | MXSITEID | MXCREATIONSTATE | ROWSTAMP | MXSTATUS | PARENT_GLOBALID | LOCATION_GLOBAL_ID | DATA_LOAD_ID | DISCIPLINE | GPS_X | GPS_Y | GPS_Z | OPERATOR | MAINTAINER | MUNICIPALITY | DISTRICT | CONTRACT_ID | PROJECT_ID | ASSET_TAG | ASSET_PRIORITY | SHAPE | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 16627 | SGNDVALV | None | None | None | None | NaN | NaN | None | None | None | None | None | None | 1996-03-09 | None | NaT | None | None | None | NaT | None | NaT | None | None | Ashghal | None | 0.0 | MUSEUM/1/V1 | Opertaional | 0.0 | 0.0 | 0.0 | NaN | NaN | 1 | {764450DA-4672-4FC1-8C26-C05E3CFEF3E7} | 650999 | None | 101 | None | 55001580 | OPERATING | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"x": 233480.25999999978, "y": 392830.72000000... | ||||||||||||
| 1 | 16624 | SGNDVALV | None | None | None | None | NaN | NaN | None | None | None | None | None | None | NaT | None | NaT | None | None | None | NaT | None | NaT | None | None | Ashghal | None | 0.0 | SW2/6/1/V1 | Opertaional | 0.0 | 0.0 | 0.0 | NaN | NaN | 1 | {9D5DAA93-C77E-48AB-80CF-5D7BCE068014} | 651002 | None | 101 | None | 55001577 | OPERATING | None | None | None | None | None | None | None | None | None | None | None | None | None | None | None | {"x": 225332.43800000008, "y": 393892.27099999... |
SG_Water_Valve.shape
(349, 71)
SG_Water_Valve_null = pd.DataFrame(SG_Water_Valve.isnull().sum())
SG_Water_Valve_null
| 0 | |
|---|---|
| OBJECTID | 0 |
| GFCODE | 0 |
| PROJECT_CODE | 49 |
| PROJECT_STATUS | 50 |
| RD_ANAME | 280 |
| ... | ... |
| CONTRACT_ID | 349 |
| PROJECT_ID | 349 |
| ASSET_TAG | 349 |
| ASSET_PRIORITY | 349 |
| SHAPE | 0 |
71 rows × 1 columns
x = SG_Water_Valve.columns
y = SG_Water_Valve.isnull().sum()
plt.figure(figsize=(30,15))
plt.bar(x, y, color= (0.1,0.2,0.8,0.2), edgecolor='Blue')
plt.suptitle("Surface Ground Water Valve Chamber", fontsize=18)
plt.ylabel("Number of Records Not filled (Null/Empty) ", fontsize=12)
plt.xticks(rotation=90, fontsize=12)
plt.show()